Probability and Statistics

Welcome to AAOC C111

Text book : Johnson and Gupta : Miller and
Freund’s Probability and Statistics for
Engineers.
Hand outs : To be distributed in tutorials.
Evaluation : 2 tests (20% each)
Tutorials (20%)
Compre. (40%)
Problems for practice : to be posted on FDIII
notice board
Also can check cmt website of the course
through Intrabits.
• For many experiments in practice, we
may not be able to tell precise outcome
of the experiment due to some
uncertainties, Statistics deals with such
experiments.
• One considers various possible
outcomes for such experiments.
 Sample spaces and events
• Definition : A sample space S of an
experiment is the set of all possible
outcomes of that experiments.
• Definition : Any subset A of the sample
space S of an experiment is called an event.
This allows us to perform various set theoretic
methods to represent sample spaces and
events.
 Examples
• Toss a coin successively 3 times and record the
upper faces (H for head and T for tail) in
sequential order.
This leads to sample space
S = {HHH, HHT, HTH, HTT, THH, THT, TTH,
TTT}
As events we can consider subsets, e.g., empty set Φ,
S, {HHH, HHT} are events.
Another event is that ‘head occurs twice’ which is
same as {HHT, HTH, THH}
 More examples

• Toss a dice and record the upper face. Here

S= {1,2,3,4,5,6}
• Toss a dice and record Y if upper face is 5
and N otherwise. Here S ={Y, N}
• Of the 3 solid chrystal and 2 CO2 lasers we
are interested in number of lasers of each
type suitable for given job. We may record
outcomes as pairs (a, b) where a= no. of
solid chrystal lasers suitable for given job,
b= no. of CO2 lasers suitable for given job.
Here S = {(a, b) : a= 0,1,2,3, b=0,1,2}.
If R : equally many lasers of both type are
suitable for job is an event then R={(0,0),
(1,1),(2,2)}
• The sample spaces above are finite, i.e.
N(S) = the cardinality of S is finite.
• Example : Record the number of tosses
of a coin required to get the tail for the
first time.
• Here the sample space S={1, 2, …},
the set of all positive integers.
• A sample space is said to be countably
infinite if it is in 1-1 correspondence
with the set of all positive integers.
• Definition : A sample space is called
discrete if it is finite or countably
infinite.
• Definition : A sample space is called
continuous if it is in 1-1
correspondence with all points of a
line, or all points of a line segment, or
all points of a plane etc.
Example 1: The temperature is recorded
in celcius in Pilani on 1-1-2006 at 8.30
am.
Here sample space S = The set of all
reals.
Example 2 : The experiment is same as
previous example except we know that
in Pilani, temp. never exceeds 50 0C
and never goes below 0 0C.
Then S = [0, 50]
Both are continuous sample spaces.
Definition : Two events C, E in a sample
space are called mutually exclusive if E∩C
is an empty set.
Extending to n events: Events A1, A2,…, An
are mutually exclusive if for any i≠j

Ai ∩ A j = Φ
Definition : The complement of an event A in
a sample space S is an event
A = { x ∈ S : x ∉A}.
• Double complements :

The complement of the complement of A is

A.
Operations on events : Like sets, we can
consider union, intersection,
complements of events. They can be
conveniently demonstrated by Venn
diagrams.
• De Morgan’s laws :

(1) ( A ∪ B ) = A ∩ B

(2) ( A ∩ B ) = A ∪ B
• Distributive laws :

(1) A ∪ ( B ∩ C ) = ( A ∪ B) ∩ ( A ∪ C )
(2) A ∩ ( B ∪ C ) = ( A ∩ B ) ∪ ( A ∩ C )
 Counting
• Multiplication rule : If sets A1, A2, …, Ak
contain respectively n1, n2, …, nk elements
then the number of ways to select an
element of A1, followed by an element of
A2 … followed by an element of Ak is
n1n2…nk.
• In general, we may have n1 choices for 1st
selection, n2 choices for 2nd selection for
each choice of 1st selection etc. Then also
• Tree diagram : Y1,1
x1 Y1,2

Y1,n2
x2

Yn1,1

xn1 Yn1,2

Yn1,n2
• Applications of multiplication Rule :
• Number of permutations of any r objects
from given n distinct objects
= nPr= (n!)/((n-r)!).
• Number of ways of choosing r objects
without repetition from n given objects=



n
=
n!
 
r 

r!( n −r )!
• If a process is broken into several steps such
that first step has n1 outcomes, for every
outcome of first step, second step has n2
outcomes and so on, then we can use
multiplication rule to find total number of
sequences of outcomes of various steps.
• Ex. 3.16 : A building inspector has to
check the wiring in a new apartment
building either on Monday, Tuesday,
Wednesday or Thursday at 8 A.M., 1
P.M. or 2 P.M. Draw tree diagram
which shows the various ways to
schedule the inspection of wiring of the
new apartment building.
 If among n objects k are alike and others are
all distinct then the number of permutations
of these n objects taken all together are
n!/k!.
(Hint : For each such permutation, substitute
for k alike objects k distinct)
• Example 3.24 (b) : In how many ways
can the television director fill 6 time
slots allocated to commercials if there
are 4 different commercials of which a
given one is to be shown 3 times and
each of the others is to be shown only
once?
Ans : (6!) / (3!).
Example 3.28 : In a carton of 12 rechargeable
batteries suppose 2 of the batteries are
defective. In how many ways can the
inspector choose 3 of the batteries and get
one of the defective batteries?

Ans : 10   2 
   
 2  1 
• Number of ways to arrange n not
necessarily distinct elements from
given N distinct elements equals Nn.
• Exercise 3.21 : In a geology class, 4
students must write a report on one of
8 field trips. In how many ways can
they choose one of the field trips if
there is no restriction on their choice?
If no 2 students can choose same field
trip?
Classical concept of Probability :
Under the assumption that all outcomes of a
finite sample space are equally likely, if s
outcomes are favourable or success out of n
possible outcomes, then probability of
success is s/n.
Equivalently, for any event A, the probability
of A is given by
N ( A)
p ( A) =
N (S )
 Phrases indicative of
Classical Probability

• randomly selected
• well shuffled
• without bias
• evenhandedly chosen etc.
Also
• fair coin
• balanced dice.
• Thus if the experiment is performed in such
a manner that all outcomes are equally
likely, then to find probability of an event
one has to count favourable outcomes and
total number of outcomes.
Exercise : From a pack of cards, one by one
choose randomly 4 cards with replacement.
What is the probability that they have
exactly 2 spades?

One needs to identify sample space and event.

See how classical probability assumptions
hold on them.
Experiment chooses 4 cards with
replacement. If we keep the record of
order in which they are selected, the
sample space is set of ways of
arranging 4 not necessarily different
cards from 52 distinct cards. It has 524
Elements. (Explain why?)
Thus event A = The set of such
arrangements in sample space such that
they have exactly 2 spades.
Also, since 4 cards are randomly
selected, we are choosing one element
of sample space at random and want to
see if it is success, classical probability
can be applied. Hence
P(A)=N(A)/N(S)
To count elements of A, we can get all
elements of A by a process containing
3 steps:
2) Determine 2 slots out of 4 for spade.
3) In the spots for spade arrange 2 (not
necessarily distinct) spade cards.
4) In the remaining cards arrange 2 not
necessarily distinct non-spade cards.
By multiplication rule (justify why it holds),

 4 2 2
N ( A) =   13 39
 2
Example 3.31 : If we roll a pair of balanced
dice, what is the probability of getting 8 or
10?
Here we mean sum on upper faces is 8 or 10.

How to identify sample space and the event

So that classical concept can be used?

Possible sums = {2,3,…, 12}?

Desirable outcomes = {8, 10}?
These won’t give correct answer, as all
outcomes of the sample space are not
equally likely, the dice are balanced
means upper faces of 2 dice are equally
likely, not their sums.
We take
S = {(a, b) : 1 ≤a ≤6, 1≤b≤6 are integers}
S has 36 outcomes.
A = {(a, b) ∈S : a+b =8 or a+b =10}.
We can list all elements of A, which are
5+3=8.
Since dice are balanced, p(A)= 8 / 36
Thus to use classical concept of
probability,
One should not always think of direct
observations of experiment as
outcomes of the sample space and
desirable outcomes as event. In stead
one has to see for which sample space
does the assumption of ‘equally likely
outcomes’ hold and appropriately
convert the desirable choices as an
event of this sample space.
Other concepts of probability : One
need not have assumption of equally
likelihood. In practice, to determine
probabilities we use
• Frequency interpretation : To decide
the probability of an event, we repeat
the experiment large number of times
independently.
p = f / n n= no. of trials, f=
no. of times the event occurred in n
trials, for large n.
(2) Subjective or personal probability :
Depends on personal judgement. These are
related to
• Odds : If probability of an event is p then
the odds for that event is the ratio p to (1-p).
• Generally odds are given as ratio a to b
where a > b are positive integers. We say
odds against an event is a to b for integers a
> b if a/b is the ratio (1-p)/p where p is the
probability of that event.
• From the feeling that odds for an event is a
to b, one gets subjective probability of the
event as p = a / (a+b).
Exercise 3.59 (a) : If a businessman feels
odds are 3 to 2 that a new venture will
succeed (I.e. he will bet $300 against
$200 for it to succeed) then his
subjective probability that the new
venture will succeed is 3/5.
General (axiomatic) probability : All the
concepts of probability ( except
subjective) can be simultaneously
studied by considering certain common
properties for axioms.
• Any properties proved using these
axioms hold for any kind of probability,
any concept which satisfies these axioms.
• Here onwards all the results for
probability are proved for this axiomatic
probability.
Set functions : A set function on a finite
sample space S is a rule which assigns
to each subset A of S a real number.

For example, a rule which assigns to

subset A of S the number N(A) is a set
function.
Axiomatic definition of probability :
For any nonempty finite set S, a set
function p on S is called a probability
if it satisfies the following axioms :
• 0 ≤ p(A) ≤ 1 for any event A of S.
• P(S)=1.
• If A and B are mutually exclusive
events in S then p(A ∪ B) = p(A) +
p(B). (additivity of set function).
Theorem : Classical probability on a finite
sample space satisfies axioms of
probability.

Clearly if for p(A) =N(A)/N(S), 0 ≤ p(A) ≤

1 and p(S) = 1. Also if A and B are
mutually exclusive then
N(A∪ B) = N(A) + N(B), so
P(A∪ B) = p(A) + p(B).
• Subjective probability need not always
satisfy 3rd axiom of probability. If it
does, then we say it is consistent and
take it seriously.
• If we are given that p is a probability
defined axiomatically, then we are not
required to prove axioms for p as we
assume axioms hold for p.
• If p is given by some other rule ( like
classical probability) then only after
proving the axioms for p can we say p
is a probability in axiomatic sense.
Theorem : p(Φ) = 0.
(Note that Theorem states that the
conclusion holds for any probability
satisfying the axioms. Thus result must
be derived only using axioms and valid
logic).
Proof : Let A = B = Φ. Then A∪B = Φ. Also
A and B are mutually exclusive, since
A ∩B = Φ. Thus by 3rd axiom of probability,
P(A∪B )= P(A) + P(B).
Hence P(Φ) = P(Φ) + P(Φ).
Subtracting P(Φ) from both sides, we get the
result.
• The empty set is called an impossible
event, event that can never occur. Thus
theorem says if an event never occurs,
its probability is 0.
• What can we say about converse? If an
event has probability 0 then can we say
it can never occur?
• With axiomatic concept of probability,
converse is not true.
• Example : Let S={1,2}, Consider the
following set function P :
P(Φ) = P({2})=0, P({1}) = P(S)=1.
P satisfies all axioms of probability.
P({2})=0 but {2} is nonempty.
Theorem : P(A) = 1 –P(A) for any event A.
Proof : ??
Theorem : If n > 1 is an integer and A1, …,
An are mutually exclusive then
P(A1∪…∪An) = P(A1) + … + P(An).
Proof : By induction on n.
Corollary : If A = { a1, …, ak} then
P(A)=p({a1}) + … + P({ak}) .

In particular,
If S={a1, …, an} then
p({a1}) + … + p({an}) =1.
Example 3.37 : An experiment has 4
possible mutually exclusive outcomes
A, B, C, D. Is the following
assignment of probabilities
permissible?
(b) p(A)= 0.28, p(B) = 0.26, p(C) =0.21,
p(D)=0.25.
(c) p(A)=0.41, p(B)=0.27, p(C)=0.28,
P(D)=0.20.
Theorem ( General addition rule) : For
any events A, B of a sample space S,
P(A∪B) = P(A) + P(B) – P(A∩B)
Idea of Proof : We use
A∪B = (A∩B)∪( A∩B)∪( A∩B)
And 3 events on RHR are mutually
exclusive.
(Verify this)
• It follows from the above that
P(A∪B) ≤ P(A) + P(B). If A, B are
mutually exclusive then equality holds.
Generalization to 3 events : P(A∪B∪C)
= P(A) + P(B) + P(C) – P(A∩B)-P(A∩
C) –P(B∩C) + P(A∩B∩C).
Idea of Proof : Let D= B∪C.
LHS= P(D ∪C) = P(D) + P(C) – P(D ∩
C)
= P(A∪B) + P(C) – P((A∩C)∪ (B∩C))
Now apply previous result to 1st and last
term.
Generalization to n events : For the
events A1, …, An of the sample space,

n n
P( Ai ) = ∑ P ( Ai ) − ∑ P( A ∩ A )i j
i =1 i =1 1≤i < j ≤ n

+ ∑ P( A ∩ A
1≤i < j < k ≤ n
i j ∩ Ak ) + ...

+ (−1) n −1 P ( A1 ∩ ... ∩ An )
Exercise 3.41 : Show that P(A∩B) ≤
P(A).
Exercise 3.53 : If A and B are mutually
exclusive then P(A) ≤ P(B).
Both of these follow from :
If C ⊆ D then P(C) ≤ P(D).
Note that D = (C∩D)∪C and these 2
events are mutually exclusive.
Thus P(D) = P (C∩D) + P(C) ≥ P(C) as
the other term is nonnegative.
Exercise 3.52 (b): If A, B are 2 events
such that P(A)=3/4 and P(B)=5/8 then
show that 3/8 ≤ P(A∩B) ≤ 5/8.

From previous ex., P(A ∩B) ≤ P(B)

=5/8.
P(A)=1/4, P(B)= 3/8. Thus P(A∪B)
≤ 1/4 + 3/8 = 5/8. By taking
complement, we get the result.
Conditional probability :
• Many a times we have to consider
events and their probabilities when
certain condition on outcomes is
already given.
• Thus whole sample space is not of
interest, only those outcomes which
satisfy given condtion can then be
taken as sample space.
• This has an effect of restricting sample
space and considering the event when
the sample space is smaller. The
probability may be different as a result.
• When we are considering probability
of an event, the total sample space is
important and must be given
consideration.
Definition : Let A, B be events in a
sample space S with P(B) > 0. The
conditional probability of A given B is
defined to be :
P(A|B) = P(A∩B) / P(B).