ISYE 3232C Fall 2015 Quiz 9

I,
, do swear that I abide by the Georgia Tech Honor Code. I understand that
any honor code violation will result in a zero for this test and one letter grade reduction.
Signature:
1. (3) True or False.
(T/F) In a MC with finite state space, all states can be transient. (F)
(T/F) A transient state is not accessible from any recurrent state. (T)
(T/F) At least one, possibly more, recurrent states are accessible from a given transient state. (T)
(T/F) A recurrent state is accessible from all states in its class, and it is also accessible from
recurrent states in other classes. (F)
(T/F) If DTMC is periodic with period d 2, then the limiting probability exist. (F)
(T/F) A DTMC can have infinite number of stationary distributions. (T)
2. Consider a DTMC on state space S = {1, 2, 3, 4} with transition probability
1 0.2
2 0
P= 3 0
4 0

0.2
1
0
0

0.4
0
0.7
0.5

0.2
0

0.3
0.5

(a) (1) Draw the transition diagram.

0.5

0.2
0.2

0.2

0.5
0.4
0.3

3
0.7

(b) (1.5) Identify the classes and classify them as recurrent or transient.
{1}: Transient
{2}: Recurrent
{3, 4}: Recurrent
(c) (0.5) What is the period of state 3?
d(3) = 1

(d) (4) Assume

?
P =
?
?

?
?
?
?

?
?
?
?
,
?
?
? 0.375

Fill the P .
Note that

f1,{3,4} = P(x {3, 4}|x0 = 1)

4

P(x {3, 4}, x1 = j|x0 = 1)

j=1

= 0.2 f1,{3,4} + 0.4 1 + 0.2 1

f1,{3,4} =

3
4

Moreover
f1,2 + f1,{3,4} = 1
Thus f1,2 =

1
4

Therefore
0 41
0 1
P =
0 0
0 0

3
4

0.625
0
0.625
0.625

3
4

0.375

0
,
0.375
0.375