Chapter 2: Probability

Shilpa G./Prasanna K.
• Course Number : MATH F113.
• Course Title: Probability & Statistics.
• Instructor-In-charge: Shilpa Gondhali.
• Instructors: Gauranga Charan Samanta, Prasanna Kumar N.
• Tutorial Instructors : Shah Parth, Jai Tushar, Karim Mosani,
Abhishek Yadav.
• FD TAs: Apoorv Jha(2016B4A70548G), Padmanabhan
Murli(2017B4A70403G), Harshavardhan K(2017B4A70601G).
• Text Book: Devore, J. L.: Probability Statistics for
Engineering and the Sciences, 8th edition, Cengage Learning,
2012.
• Reference Books:
1] Milton, J. S. and Arnold J. C.: Introduction to Probability
and Statistics: Principles and Applications for Engineering and
the Computing Sciences, 4th edition, Tata McGraw-Hill, 2007.
• Reference books (Cont...):
2] Johnson, R. A., Miller: Freunds Probability and Statistics
for Engineers, 8th edition, PHI, 2010.
3] Meyer, P. L.: Introductory Probability and Statistical
Applications, 2nd edition, Addison-Wesley, 1970.
4] Ross, S. M.: Introduction to Probability Models, 11th
edition, Academic Press, 2014.
• Reference books (Cont...):
5] Walpole, R. E., Myers, R. H., Myers, S. L., Ye, K. E.:
Probability Statistics for Engineers and Scientists, 9th
edition, Pearson Education, 2016.
 Evaluation Scheme

Comp. Duration Details Marks Rem.

Midterm 90 mins RTH 90 CB
Quiz 1 *** 07/09/19 (ten- 45 OB
tative)
Quiz 2 *** 09/11/19 (ten- 45 OB
tative)
Compre 3 Hours RTH 120 CB
RTH= Refer to Handout I
Pierre de Fermat Blaise Pascal

Girolamo Cardano
 (Modern) Prob & Stats in India

• Mahalanobis
• C. R. Rao
• Famous Four: K. R. Parthasarathy, R. Ranga Rao, Veeravalli
S. Varadarajan, S. R. Srinivasa Varadhan.
Definition
A sample space for an experiment is a set S with the property that
each physical outcome of the experiment corresponds to exactly
one element of S. An element of S is called a sample point.
Any subset A of a sample space is called an event.
 Methods for assigning probability to an event

• Personal approach,
• Classical approach,
• Relative frequency (statistical approach).
 Probability of occurrence of an event A

• Classical formula

n(A)
P[A] :=
n(S)
number of ways A can occur
=
no. of ways the expt. can proceed
Example
A committee of 5 is to be selected from a group of 6 men and 9
women. If the selection is made randomly, what is the probability
that the committee consists of exactly 3 men and 2 women?
(63)(92) 240
Answer: = 1001 .
(155)
 Probability of occurrence of an event A

• Relative frequency approximation

f
P[A] :=
n
number of times A occured
=
no. of times expt. was run
Example
If records show that 294 of 300 ceremic insulators tested were able
to withstand a certain thermal shock, what is the probability that
any one untested insulator will be able to withstand the thermal
shock?
294
Answer: 300 .
Definition
Two events A1 and A2 are mutually exculsive iff A1 ∩ A2 = Ø.
Events A1 , A2 , . . . are mutually exclusive iff Ai ∩ Aj = Ø for all
i 6= j.
 Need of another definition of probability

1. If we pick a number in [0, 1] randonly then what is a chance

that the number picked would be a rational number?
2. What is the probability that randomly picked integer is an
even integer?
 Axioms of probability

• Let S denote a sample space for an experiment. Then,

P[S] = 1.
• P[A] ≥ 0 for every event A.
• Let A1 , A2 , . . . be a finite or an infinite collection of mutually
exclusive events. Then,

P[A1 ∪ A2 ∪ · · · ] = P[A1 ] + P[A2 ] + · · ·

Theorem
• P[Ø] = 0.
• For an event A, P[Ac ] = 1 − P[A].
• P[A1 ∪ A2 ] = P[A1 ] + P[A2 ] − P[A1 ∩ A2 ].
Problem
An ATM personal identification number (PIN) consists of four
digits, each a 0, 1, 2, ...8, or 9, in succession.
a. According to a representative at a local branch of SBI, there are
in fact restrictions on the choice of digits. The following choices
are prohibited: (i) all four digits identical (ii) sequences of
consecutive ascending or descending digits, such as 6543 (iii) any
sequence starting with 19 (birth years are too easy to guess). So if
one of the PINs is randomly selected, what is the probability that it
will be a legitimate PIN (that is, not be one of the prohibited
sequences)? Ans.: .9876.
Problem (Cont...)
b. Someone has stolen an ATM card and knows that the first and
last digits of the PIN are 1 and 1, respectively. He has three tries
before the card is retained by the ATM (but does not realize that).
So he randomly selects the 2nd and 3rd digits for the first try, then
randomly selects a different pair of digits for the second try, and
yet another randomly selected pair of digits for the third try (the
individual knows about the restrictions described in (a) so selects
only from the legitimate possibilities). What is the probability that
3
the individual gains access to the account? Ans.: 89 .
Problem
The composer Beethoven wrote 9 symphonies, 5 piano concertos
(music for piano and orchestra), and 32 piano sonatas (music for
solo piano).
a. How many ways are there to play first a Beethoven symphony
and then a Beethoven piano concerto? Ans.: 45.
b. The manager of a radio station decides that on each successive
evening (7 days per week), a Beethoven symphony will be played
followed by a Beethoven piano concerto followed by a Beethoven
piano sonata. For how many years could this policy be continued
before exactly the same program would have to be repeated? Ans.:
1440 days. That is, almost 4 years.
 Example-

Problem (1)
Two dice are rolled. Let X1 and X2 denote number appearing on
1st and 2nd die (resp.) after the roll. Let W = max {X1 , X2 }. Is
probability of (W = 3) same as probability of (W = 4)?

5 7
Answer: P[W = 3] = 36 , P[W = 4] = 36 .
 Example

Problem (2)
If we roll two fair dice, which is more likely: a sum of 11 or a sum
of 12?

Solution: There are 36 possible outcomes for ordered pairs of the

form (value of A, value of B), and they are equally likely by
symmetry. Of these, (5, 6) and (6, 5) are favorable to a sum of 11,
while only (6, 6) is favorable to a sum of 12. Therefore a sum of
11 is twice as likely as a sum of 12; the probability is 1/18 for the
former, and 1/36 for the latter!
 Example

Problem (3)
There are 100 passengers lined up to board an airplane with 100
seats (with each seat assigned to one of the passengers). The first
passenger in line crazily decides to sit in a randomly chosen seat
(with all seats equally likely). Each subsequent passenger takes his
or her assigned seat if available, and otherwise sits in a random
available seat. What is the probability that the last passenger in
line gets to sit in his or her assigned seat?
Solution:The probability that the last passenger gets seat 100 is
1/2.
 Example-Birthday problem
Problem (4)
There are k people in a room. Assume each person’s birthday is
equally likely to be any of the 365 days of the year (we exclude
February 29), and that people’s birthdays are independent (we
assume there are no twins in the room). What is the probability
that two or more people in the group have the same birthday?

365.364 · · · (365 − k + 1)
Solution: P(no birthday match) = , and
365k
the probability of at least one birthday match is
365.364 · · · (365k + 1)
P(at least 1 birthday match) = 1 − .
365k
An interesting observation. The first value of k for which the
probability of a match exceeds 0.5 is k = 23. Thus, in a group of
23 people, there is a better than 50% chance that two or more of
them will have the same birthday. By the time we reach k = 57,
the probability of a match exceeds 99%.
 Thinking conditionally

• Probability is a language for expressing our degrees of belief or

uncertainties about events.
• Whenever we observe new evidence (i.e., obtain data), we
acquire information that may affect our uncertainties.
• Conditional probability is the concept that addresses this
fundamental question: how should we update our beliefs in
light of the evidence we observe?
 Thinking conditionally

• Conditional probability is essential for scientific, medical, and

legal reasoning, since it shows how to incorporate evidence
into our understanding of the world in a logical, coherent
manner.
• Most probabilities are conditional ; whether or not its written
explicitly, there is always background knowledge (or
assumptions) built into every probability.
• Conditioning is the soul of statistics!
 Conditional probability

Definition
Let A1 and A2 be events such that P[A1 ] 6= 0. The conditional
probability of A2 given A1 , denoted by P[A2 |A1 ], is defined by

P[A1 ∩ A2 ]
P[A2 |A1 ] := .
P[A1 ]

Here A2 is the event whose uncertainty we want to update, and A1

is the evidence we observe (or want to treat as given). We call
P[A2 ] the prior probability of A2 and P[A2 |A1 ] the posterior
probability of A2 .
Example
Problem (5)
70% of your friends like Chocolate, and 35% like Chocolate and
like Strawberry. What percent of those who like Chocolate also like
Strawberry?

Answer:
0.35
P(Strawberry/Chocolate) = 0.7 = 50%.
 Example

Problem (6)
Two dice are rolled. If the first one top with 5 then find the
probability that the total of the two will be greater than 7.

Answer: Let A be the event of getting a 5 on first die. Then

P[A] = 1/6. Let B be the event of getting the total greater than 7.
A ∩ B = {(5, 3), (5, 4), (5, 5), (5, 6)}. Hence,
P[A ∩ B] = 4/36 = 1/9. So,

2
P[B|A] = .
3
 Example

Problem (7)
A standard deck of cards is shuffled well. Two cards are drawn
randomly, one at a time without replacement. Let A be the event
that the first card is a heart, and B be the event that the second
card is red. Find P[A|B] and P[B|A].
Ans: P(A) = 1/4 and P(B) = 1/2, and P(A ∩ B) = 25/204.
We now have all the pieces needed to apply the definition of
conditional probability:

P(A|B) = P(A ∩ B)/P(B) = (25/204)/(1/2) = 25/102,

and

P(B|A) = P(B ∩ A)/P(A) = (25/204)/(1/4) = 25/51.

 Points worth noting...

• P(A|B) 6= P(B|A). If instead we had defined B to be the

event that the second card is a heart, the two conditional
probabilities would have been equal.
• Both P(A|B) and P(B|A) make sense (intuitively and
mathematically).
• We can also see that P(B|A) = 25/51 by a direct
interpretation of what conditional probability means: if the
first card drawn is a heart, then the remaining cards consist of
25 red cards and 26 black cards (all of which are equally likely
to be drawn ), so the conditional probability of getting a red
card is 25/(25 + 26) = 25/51.
• It is harder to find P(A|B) in this way!
 Prove and apply for the problems...
• P(A|S) = P(A)
• If B ⊂ A and P(B) 6= 0, then P(A|B) = 1.
• If A ∩ B = φ and P(B) 6= 0, then P(A|B) = 0.
• If A ∩ B = φ and P(C ) 6= 0, then
P((A ∪ B)|C ) = P(A|C ) + P(B|C ).
• For any events A and B with positive probabilities,

P(A ∩ B) = P(A|B)P(B) = P(B|A)P(A).

• In general, we have

P[E1 ∩ E2 · · · ∩ En ] =
P[E1 ]P[E2 |E1 ]P[E3 |E1 ∩ E2 ] · · · P[En |E1 ∩ · · · ∩ En−1 ].

• If P(A) 6= 0, and P(B) 6= 0 then P(A/B) ≥ P(A), is

equivalent to P(B/A) ≥ P(B).
 Example

Problem (8)
A student has to sit for an examination consisting of 3 questions
selected randomly from a list of 100 questions. To pass he needs
to answer all the questions. What is the probability that the
student will pass the examination if he knows the answers to 90
questions on the list?
Ans. Let the questions be numbered 1,2,3. Let Ai be the event
that question i is among those the student can answer. We need
to find P(A1 ∩ A2 ∩ A3 ). But
P(A1 ∩ A2 ∩ A3 ) = P[A1 ]P[A2 |A1 ]P[A3 |A1 ∩ A2 ] = 0.73
 Law of total probability and Bayes’ theorem

Theorem (Law of total probability)

Let A1 , A2 , . . . , An be a collection of mutually exclusive events
whose union is S with P(Ai ) > 0 for all i. Then
n
X
P[B] = P[B|Ai ]P[Ai ].
i=1
 Example

Problem (9 )
Seventy percent of the light aircraft that disappear while in flight
in a certain country are subsequently discovered. Of the aircraft
that are discovered, 60% have an emergency locator, whereas 90%
of the aircraft not discovered have such a locator. Suppose a light
aircraft has disappeared.
a. If it has an emergency locator, what is the probability that it
will not be discovered?
b. If it does not have an emergency locator, what is the probability
that it will be discovered?
Ans. Let D be the event that an aircraft is discovered and L be the
event that aircraft discovered has an emergency locator. We need
to find (a) P(D c |L) and (b) P(D|L0 ) where L0 = D − L
Ans. We have P(D) = 0.7, P(L|D) = 0.6 and P(L|D c ) = 0.9.
P(L ∩ D) = 0.6.0.7 = 0.42 and P(L ∩ D c ) = 0.3.0.9 = 0.27.
a.
P(L) = P(L/D)P(D) + P(L/D c )P(D c ) = 0.6.0.7 + 0.9.0.3 = 0.69.
c ∩L)
Therefore P(D c |L) = P(DP(L) = 0.27/0.69 = 0.3913.
0
b. P(D ∩ L ) = P(D) − P(D ∩ L) = 0.7 − 0.42 = 0.28, and
P(L0 ) = 1 − P(L) = 1 − 0.69 = 0.31. So
0)
P(D|L0 ) = P(D∩L
P(L0 ) = 0.28/0.31 = 0.9032.
 Bayes’ Theorem

Theorem
Let A1 , A2 , . . . , An be a collection of mutually exclusive events
whose union is S. Let B be any event such that P[B] 6= 0. Then
for any Aj , j = 1, · · · , n, with P(Aj ) > 0

P[B|Aj ]P[Aj ]
P[Aj |B] =
P[B]

where P[B] is as given by Law of total probability, i.e.,

n
X
P[B] = P[B|Ai ]P[Ai ].
i=1
 Example

Problem (10)
A spam filter is designed by looking at commonly occurring
phrases in spam. Suppose that 80% of email is spam. In 10% of
the spam emails, the phrase ”free money” is used, whereas this
phrase is only used in 1% of non-spam emails. A new email has
just arrived, which does mention ”free money”. What is the
probability that it is spam?
Ans: Let S be the event that an email is spam and F be the event
that an email has the ”free money” phrase.
We need to find P(S|F ), but it is easier to find P(F |S). By Bayes’
rule,
P(S|F ) = [P(F |S)P(S)]/P(F )
= (0.1.0.8)/(0.1.0.8 + 0.01.0.2) = 80/82.
 Example
Problem (11)
You have one fair coin, and one biased coin which lands Heads with
probability 3/4. You pick one of the coins at random and flip it
three times. It lands Heads all three times. Given this information,
what is the probability that the coin you picked is the fair one?
Ans. Let A be the event that the chosen coin lands Heads three
times and let F be the event that we picked the fair coin.
We are interested in P(F |A), but it is easier to find P(A|F ) and
P(A|F c ).
Doing so, we have

P(A|F )P(F ) P(A|F )P(F )

P(F |A) = =
P(A) P(A|F )P(F ) + P(A|F c )P(F c )

(1/2)3 .1/2
= = 0.23.
(1/2)3 .1/2
+ (3/4)3 .1/2
 Example

Problem (12)
A certain item is manufactured by three factories say 1,2,3. It is
known that 1 turns out twice as many items as 2, and 2 and 3 turn
out the same number of items. It is also known that 2 percent of
the items produced by 1 and by 2 are defective, while 4 percent of
those manufactured by 3 are defective. All the items produced are
put into one stockpile, and then one item is chosen at random.
What is the probability that this item is defective? what is the
probability that it was produced in factory 1?
Ans. Let A = {defective items}, B1 = {items from 1},
B2 = {items from 2}, and B3 = {items from 3}.
 Example

Problem (12)
A certain item is manufactured by three factories say 1,2,3. It is
known that 1 turns out twice as many items as 2, and 2 and 3 turn
out the same number of items. It is also known that 2 percent of
the items produced by 1 and by 2 are defective, while 4 percent of
those manufactured by 3 are defective. All the items produced are
put into one stockpile, and then one item is chosen at random.
What is the probability that this item is defective? what is the
probability that it was produced in factory 1?
Ans. Let A = {defective items}, B1 = {items from 1},
B2 = {items from 2}, and B3 = {items from 3}.
We require P(A) and P(B1 |A). We have P(B1 ) = 1/2,
P(B2 ) = P(B3 ) = 1/4, ,P(A|B1 ) = P(A|B2 ) = 0.02, and
P(A|B3 ) = 0.04.
(a) P(A) = P(A|B1 )P(B1 ) + P(A|B2 )P(B2 ) + P(A|B3 )P(B3 ).
We obtain P(A) = 0.025.
(b) P(B1 |A) = P(A|B1 )P(B1 )
P(A) = 0.40.
 Thinking ’independently’
• We have now seen several examples where conditioning on
one event changes our beliefs about the probability of another
event. The situation where events provide no information
about each other is called independence.
• Alternatively, A and B are independent if learning that B
occurred gives us no information that would change our
probabilities for A occurring (and vice versa).
• Note that independence is a symmetric relation: if A is
independent of B, then B is independent of A.
• Independence is completely different from disjointness. If A
and B are disjoint, then P(A ∩ B) = 0, so disjoint events can
be independent only if P(A) = 0 or P(B) = 0. Knowing that
A occurs tells us that B definitely did not occur, so A clearly
conveys information about B, meaning the two events are not
independent.
 Thinking ’independently’
• If A provides no information about whether or not B occurred,
then it also provides no information about whether or not B c
occurred.
• Events A and B are independent iff P(A|B) = P(A) iff
P(B|A) = P(B).
• Definition
Events A1 and A2 are independent if P[A1 ∩ A2 ] = P[A1 ] P[A2 ].
• Let C = {Ai |i = 1, . . . , n} be a finite collection of events.
These events are independent iff given any subcollection
A(1) , . . . , A(m) of elements of C ,
m
Y
P[A(1) ∩ A(2) ∩ · · · ∩ A(m) ] = P[A(i) ].
i=1
Proposition. If A and B are independent, then A and B c are
independent, Ac and B are independent, and Ac and B c are
independent.
 Thinking ’independently’
• Pairwise independence doesnt imply independence.
• Consider two fair, independent coin tosses, and let A be the
event that the first is Heads, B the event that the second is
Heads, and C the event that both tosses have the same result.
Then A, B, and C are pairwise independent but not
independent, since P(A ∩ B ∩ C ) = 1/4 while
P(A)P(B)P(C ) = 1/8. The point is that just knowing about
A or just knowing about B tells us nothing about C, but
knowing what happened with both A and B gives us
information about C (in fact, in this case it gives us perfect
information about C).
• On the other hand, P(A ∩ B ∩ C ) = P(A)P(B)P(C ) does not
imply pairwise independence; this can be seen quickly by
looking at the extreme case P(A) = 0, when the equation
becomes 0 = 0 and tells us nothing about whether B and C
are independent.
 Example

Problem (13)
A family has 3 children, creatively named A, B, and C.
(a) Discuss intuitively (but clearly) whether the event ”A is older
than B” is independent of the event ”A is older than C”.
(b) Find the probability that A is older than B, given that A is
older than C.
Ans: (a) Not independent: The fact that ’A is older than B’ makes
it more likely that A is older than C.
(b) 2/3?
Problem (14)
Is it possible that an event is independent of itself? If so, when is
this the case?
Ans. Possible only when the event has probability 0 or 1.
 Example
Problem (15)
An oil exploration company currently has two active projects, one
in Asia and the other in Europe. Let A be the event that the Asian
project is successful and B be the event that the European project
is successful. Suppose that A and B are independent events with
P(A) = .4 and P(B) = .7. (a) If the Asian project is not
successful, what is the probability that the European project is also
not successful? Explain your reasoning. (b) What is the probability
that at least one of the two projects will be successful?
Ans. a. P(B 0 ) = 1 − P(B) = 0.3
b. P(A ∪ B) = P(A) + P(B) − P(A ∩ B) =
P(A) + P(B) − P(A)P(B) = 0.82.
Problem (16)
If A, B, C are independent events show that A, B ∩ C are
independent and also A, B ∪ C are independent.
THANQ