2.

1 Exercises
1. Explain the role of uncertainty (epistemic uncertainty) and variability (aleatory uncertainty) in the
context of bayesian inference by using some example (other than the lecture examples).

Growth of bacteria: initial conditions could include important factors such as temperature. For given
temperature T , the resulting number of bacteria after time t could be Nt , which could be described by
some model P (Nt | T ) (aleatory uncertainty), and we could specify also π(T ) (epistemic uncertainty).
Note that π(T ) could be based on some earlier data describing the variation of temperature in fridges
of dierent households. And this distribution could then be specied as π(T | θ). Then, we could
describe epistemic uncertainty with some π(θ). But this again could be based on some data about
variability between households in dierent counties. It seems dicult to justify the claim that any
of these distributions would be expressing the 'true variability'. In bayes these are all conditional
uncertainties, depending on the background information.

2. In the example of eliciting your subjective probability of event A you were oered to choose between
a (1) lottery ticket P (win) = n/100 and a (2) possible reward of same value if event A occurs. What is
the expected value Ei of your gain if you choose option i? (i ∈ {1, 2}). Solve P (A) from the equation
E1 = E2 .

E1 = M n/100 + 0(100 − n)/100

E2 = M P (A) + 0(1 − P (A))

3. Grandfather goes shopping in four shops. In each shop, he can forget his umbrella with probability
0.25, and remembers it with probability 0.75. After he has gone through all four shops, what is the
probability that the umbrella was forgotten in the third shop? What is the probability that it was
forgotten in the third shop, given that he really did forget it in one of the shops?

P( R R R R ) = 0.754
P( R R R F ) = 0.753 × 0.25
P( R R F ) = 0.752 × 0.25
P( R F ) = 0.75 × 0.25
P( F ) = 0.25
P (forgot in third shop) = 0.752 × 0.25
0.752 × 0.25
P (forgot in third shop | forgot in one of the shops) =
0.753 × 0.25 + 0.752 × 0.25 + 0.75 × 0.25 + 0.25
4. Assume the goal is to infer about some quantity X based on observations about another quantity
Y . If the joint distribution P (X, Y ) is known, either P (Y | X) or P (X | Y ) could be derived from
this. If X is informative about Y (i.e. P (Y | X) 6= P (Y )), or vice versa, does it mean that there must
be a causal relationship between the two? Find supporting examples.

Not necessarily causal. For example: ice cream consumption and outdoor temperature may be corre-
lated, but not strictly causal. (We could drink just water instead?)

1
5. Continue Exercise 1.2. Assume there is some genotype in humans that can protect from serious
infections, and the population prevalence of this type is g = 0.03. If a person has the genotype and is
infected, the conditional probability of survival is one, regardless of the virus type. If a person does
not have the genotype, assume the conditional probabilities only depend on the inuenza type as given
in the exercise. Assume a patient survived infection that was one of the types T1 , T2 , T3 . What is the
probability that he had this advantageous genotype?

P (S | G, inf)P (G)
P (G | S, inf) = ≈ 0.03001702
P (S | G, inf)P (G) + P (S | no G, inf)P (no G)

P (G) = g = 0.03
P (S | G, inf) = 1
P (T1 ) P (T2 ) P (T3 )
P (S | no G, inf) = P (S | T1 ) P + P (S | T2 ) P + P (S | T3 ) P
P (Ti ) P (Ti ) P (Ti )
0.05 0.1 0.3
= 0.995 + 0.9999 + 0.99999 = 0.9994156
0.45 0.45 0.45
6. Approximately 1/125 of all births are fraternal twins and 1/300 identical twins. Elvis Presley had
a twin brother who died at birth. What is the probability that Elvis was an identical twin?

1 1
P (identical twins & twin brother) = P (identical twins)P (both boys | identical twins) =
300 2
1 1
P (fraternal twins & twin brother) = P (fraternal twins)P (both boys | fraternal twins) =
125 4
1 1
300 2 5
P (identical twin | twin brother) = 1 1 1 1 =
300 2
+ 125 4
11
This example was taken from Gelman book. Thanks for noting alternative idea, based on noting that

P (both boys | identical twins, Elvis was boy) = 1

which seems to lead to

1 1
300 1 5
1 1 1 1 =
300 1
+ 125 2
11
7. Posterior of r was calculated in two steps: after observing one red ball, and then using this posterior
as a new prior when the 2nd ball was observed red. Verify that we get the same posterior probability
formula by using P (r = i/N ) = 1/(N + 1) as the prior, and P (1st X = red, 2nd X = red | r = i/N ) as
the model for both observations.

i 2 1
P (r = i/N ) ∝ ( )
N N +1
Normalizing constant:
XN
1 (2N + 1)
2
i2 =
N (N + 1) i=0 6N

2
So that the posterior is:
i2 6N 6i2
=
N 2 (N + 1) (2N + 1) N (N + 1)(2N + 1)
Therefore, logically it does not matter which result we observe rst, or if we observe both results at
once. Hence, 'prior' does not necessarily mean the same as 'before in time', but only 'logically prior
to revealing a piece of data'.

8. You strongly suspect your car must have either electricity problems, or some other problems,
but not both. If the car has electricity problems, the key's remote control button won't open the
door with probability P (not open | e-problem) = 0.1. Otherwise, the probability is P (not open |
other problem) = 0.2. Then, P (e-problem) + P (other problem) = 1, and P (no problem) = 0. Now,
you push the key-button and nothing happens. What is the probability that the car really has e-
problems? Discuss how this depends on your initial beliefs about the existence of e-problems versus
other possible problems. (Plot the posterior as a function of prior). In this model, have you included
the possibility that you might have wrong keys?
1.0

P(other problems | data )

0.8
Posterior probability

0.6
0.4
0.2

P(e−problems | data )
0.0

0.0 0.2 0.4 0.6 0.8 1.0

prior P(e−problems)

Figure 1: Posterior probability, using dierent prior probabilities.

9. When dad bakes gingerbread, they get burned with probability 0.5. When mom bakes, they get
burned with probability 0.9. Gingerbread is made every month. Every other month dad is cooking,
and mom is cooking for other months. One day, their son was served with unburned gingerbread.
What is the probability, that dad was the cook? What if we know dad is cooking for one month per
year only? What prior probability would give <10% posterior probability?

P (U B | D)P (D)
P (D | U B) =
P (U B | D)P (D) + P (U B | M )P (M )
0.5 ∗ 0.5
= = 0.8333333 (prior 0.5)(post odds 5)
0.5 ∗ 0.5 + 0.1 ∗ 0.5

3
 0.5 ∗ 1/12
= 0.3125 (prior 0.08333333) (prior odds 0.09, post odds 0.45)
0.5 ∗ 1/12 + 0.1 ∗ 11/12
Posterior < 0.1 when prior < 0.01/0.46 ≈ 0.022.