Engineering Risk Benefit Analysis

1.155, 2.943, 3.577, 6.938, 10.816, 13.621, 16.862, 22.82, ESD.72, ESD.721

RPRA 2.

Elements of Probability Theory

George E. Apostolakis Massachusetts Institute of Technology

Spring 2007
RPRA 2. Elements of Probability Theory 1

Probability: Axiomatic Formulation

The probability of an event A is a number that satisfies the following axioms (Kolmogorov): 0 P(A) 1 P(certain event) = 1 For two mutually exclusive events A and B: P(A or B) = P(A) + P(B)

RPRA 2. Elements of Probability Theory

Relative-frequency interpretation
Imagine a large number n of repetitions of the experiment of which A is a possible outcome.

If A occurs k times, then its relative frequency is:

k n

It is postulated that:

k lim P( A ) n n

RPRA 2. Elements of Probability Theory

Degree-of-belief (Bayesian) interpretation

No need for identical trials. The concept of likelihood is primitive, i.e., it is meaningful to compare the likelihood of two events. P(A) < P(B) simply means that the assessor judges B to be more likely than A. Subjective probabilities must be coherent, i.e., must satisfy the mathematical theory of probability and must be consistent with the assessors knowledge and beliefs.
RPRA 2. Elements of Probability Theory 4

Basic rules of probability: Negation

S
___

Venn Diagram

EE = S

P(E) + P(E) = P(S) = 1

P(E) = 1 P(E)
RPRA 2. Elements of Probability Theory 5

Basic rules of probability: Union

N 1 N N N P U A i = P (A i ) P (A i A j ) + 1 i =1 i =1 j= i + 1

K + ( 1)

N +1

N P I A i 1

Rare-Event Approximation:

N N P P (A i ) U Ai 1 i =1
RPRA 2. Elements of Probability Theory 6

Union (contd)
For two events: P(AB) = P(A) + P(B) P(AB)

For mutually exclusive events: P(AB) = P(A) + P(B)

RPRA 2. Elements of Probability Theory

Example: Fair Die

Sample Space: {1, 2, 3, 4, 5, 6} (discrete)

Fair: The outcomes are equally likely (1/6). P(even) = P(2 4 6) = (mutually exclusive)

RPRA 2. Elements of Probability Theory

Union of minimal cut sets

From RPRA 1, slide 15 X T = M i M i M j + ... + (1)
i =1 i =1 j= i +1 N N 1 N N +1

i =1

Mi

P( X T ) =

P(Mi ) P(MiM j ) + ... + ( 1)

i =1 i =1 j = i +1

N 1 N

N +1

N P Mi i=1
N 1

Rare-event approximation:

P(XT ) P (Mi )

RPRA 2. Elements of Probability Theory

Upper and lower bounds

P( X T ) =

i =1

P(Mi ) P(Mi M j ) + ... + (1)

i =1 j=i +1
N 1

N1 N

N+1 N

i =1

P(Mi )

P(XT ) P (Mi )

The first term, i.e., sum, gives an upper bound.

N 1 N

P( X T ) P( M i )
i =1

P( M i M j )
i =1 j= i +1

The first two terms, give a lower bound.

RPRA 2. Elements of Probability Theory 10

Conditional probability
P(AB ) P(A B ) P(B )
P(AB ) = P(A B )P(B ) = P(B / A )P(A )

For independent events:

P(AB) = P(A )P(B ) P(B / A ) = P(B )

Learning that A is true has no impact on our probability of B.

RPRA 2. Elements of Probability Theory 11

Example: 2-out-of-4 System

1 2 3

M 1 = X1 X 2 X 3 M 3 = X3 X 4 X 1

M2 = X2 X3 X4 M4 = X1 X2 X4

XT = 1 (1 M1) (1 M2) (1 M3) (1 M4)

XT = (X1 X2 X3 + X2 X3 X4 + X3 X4 X1 + X1 X2 X4) - 3X1 X2 X3X4

RPRA 2. Elements of Probability Theory 12

2-out-of-4 System (contd)

P(XT = 1) = P(X1 X2 X3 + X2 X3 X4 + X3 X4 X1 + X1 X2 X4) 3P(X1 X2 X3X4) Assume that the components are independent and nominally identical with failure probability q. Then, P(XT = 1) = 4q3 3q4 Rare-event approximation: P(XT = 1) 4q3

RPRA 2. Elements of Probability Theory

13

Updating probabilities (1)

The events, Hi, i = 1...N, are mutually exclusive and exhaustive, i.e., HiHj= , for ij, Hi = S, the sample space. Their probabilities are P(Hi). Given an event E, we can always write
H3 H1 H2 E

P( E) = P( E / H i )P(H i )
1

RPRA 2. Elements of Probability Theory

14

Updating probabilities (2)

Evidence E becomes available. What are the new (updated) probabilities P(Hi/E)? Start with the definition of conditional probabilities, slide 11.
P ( EH i ) = P ( E / H i ) P ( H i ) = P ( H i / E ) P ( E )

P(H i / E) =

P ( E / H i )P ( H i ) P(E)

Using the expression on slide 14 for P(E), we get

RPRA 2. Elements of Probability Theory 15

Bayes Theorem

Likelihood of the Evidence

P (H i E ) =
Posterior Probability

P(E H i )P(H i )
1

P(E H i )P(H i )

Prior Probability

RPRA 2. Elements of Probability Theory

16

Example: Lets Make A Deal

Suppose that you are on a TV game show and the host has offered you what's behind any one of three doors. You are told that behind one of the doors is a Ferrari, but behind each of the other two doors is a Yugo. You select door A. At this time, the host opens up door B and reveals a Yugo. He offers you a deal. You can keep door A or you can trade it for door C. What do you do?

RPRA 2. Elements of Probability Theory

17

Lets Make A Deal: Solution (1)

Setting up the problem in mathematical terms: A = {The Ferrari is behind Door A} B = {The Ferrari is behind Door B} C = {The Ferrari is behind Door C} The events A, B, C are mutually exclusive and exhaustive. P(A) = P(B) = P(C) = 1/3

E = {The host opens door B and a Yugo is behind it} What is P(A/E)? Bayes Theorem
18

RPRA 2. Elements of Probability Theory

Lets Make A Deal: Solution (2)

P (E A )P (A ) = P (E A )P (A ) + P (E B )P (B ) + P (E C )P (C )
But P(E/B) = 0 (A Yugo is behind door B).

P (A E ) =

P(E/C) = 1

(The host must open door B, if the Ferrari is behind door C; he cannot open door A under any circumstances).
RPRA 2. Elements of Probability Theory 19

Lets Make A Deal: Solution (3)

Let P(A/E) = x and P(E/A) = p Bayes' theorem gives:

p x= 1+ p

Therefore

For P(E/A) = p = 1/2 (the host opens door B randomly, if the Ferrari is behind door A) P(A/E) = x = 1/3 = P(A) (the evidence has had no impact)

RPRA 2. Elements of Probability Theory

20

Lets Make A Deal: Solution (4)

Since P(A/E) + P(C/E) = 1

P(C/E) = 1 - P(A/E) = 2/3 The player should switch to door C

For P(E/A) = p = 1 (the host always opens door B, if the Ferrari is behind door A) P(A/E) = 1/2 offer any advantage. P(C/E) = 1/2, switching to door C does not

RPRA 2. Elements of Probability Theory

21

Random Variables

Sample Space: The set of all possible outcomes of an experiment. Random Variable: A function that maps sample points onto the real line. Example: For a die S = {1,2,3,4,5,6}

For the coin: S = {H, T} {0, 1}

RPRA 2. Elements of Probability Theory

22

Events
3.6
- 0 1 2 3

4 5
6

We say that {X x} is an event, where x is any number on the real line. For example (die experiment): {X 3.6} = {1, 2, 3} {1 or 2 or 3} {X 96} = S {X -62} = (the certain event) (the impossible event)

RPRA 2. Elements of Probability Theory

23

Sample Spaces

The SS for the die is an example of a discrete sample space and X is a discrete random variable (DRV). A SS is discrete if it has a finite or countably infinite number of sample points. A SS is continuous if it has an infinite (and uncountable) number of sample points. The corresponding RV is a continuous random variable (CRV). Example: {T t} = {failure occurs before t}

RPRA 2. Elements of Probability Theory

24

Cumulative Distribution Function (CDF)

The cumulative distribution function (CDF) is F(x) Pr[X x] This is true for both DRV and CRV.

Properties: 1. F(x) is a non-decreasing function of x. 2. F(-) = 0 3. F() = 1

RPRA 2. Elements of Probability Theory

25

CDF for the Die Experiment

F(x)
1

1/ 6 1 2 3 4 5 6

RPRA 2. Elements of Probability Theory

26

Probability Mass Function (pmf)

For DRV: probability mass function

P(X = x i ) p i
xi x
normalization
6 1

F(x ) = p i , for all

P( S ) = p i = 1
i

Example: For the die, pi = 1/6 and p i = 1

F ( 2 .3 ) = P ( 1 2 ) = p i =
1 2

1 1 1 + = 6 6 3

RPRA 2. Elements of Probability Theory

27

Probability Density Function (pdf)

f(x)dx = P{x < X < x+dx}

dF (x ) f (x ) = dx
P( S ) = F( ) =

F( x ) = f (s )ds

f (s)ds = 1
normalization
28

RPRA 2. Elements of Probability Theory

Example of a pdf (1)

Determine k so that

f (x ) = kx 2 ,
f (x ) = 0,
is a pdf. Answer:

for

0x1

otherwise

The normalization condition gives:

1 0 2 kx dx = 1

k=3
29

RPRA 2. Elements of Probability Theory

Example of a pdf (2)

F(x) 1 0.67 0.42

F( x) = x 3
F(0.875) F(0.75) = 3x 2dx =
0.75 0.875

1 f(x) 3

= 0.67 - 0.42 = 0.25 = = P{0.75 < X < 0.875}

1 0.75 0.875

RPRA 2. Elements of Probability Theory

30

Moments
Expected (or mean, or average) value

xf (x )dx E[X ] m x jp j j

CRV

DRV

(x m ) f (x )dx 2 2 E (X m ) = (x j m )2 p j j RPRA 2. Elements of Probability Theory

Variance (standard deviation )

CRV DRV
31

Percentiles
Median: The value xm for which

F(xm) = 0.50 For CRV we define the 100 percentile as that value of x for which

f (x )dx =
RPRA 2. Elements of Probability Theory 32

Example
m = 3x 3 dx = 0.75
0 1

2 = 3(x 0.75 ) x 2 dx = 0.0375

2 0
3 F (x m ) = x m = 0. 5

= 0.194

x m = 0.79

3 x0 .05 = 0.05 3 x0 .95 = 0.95

x 0.05 = 0.37 x 0.95 = 0.98

33

RPRA 2. Elements of Probability Theory