ISYE 3232A Fall 2014 Quiz 3 - A

I,
, do swear that I abide by the Georgia Tech Honor Code. I understand
that any honor code violations will result in a failure (an F).
Signature:
You will have 1 hour.
This quiz is closed book and closed notes. Calculators are not allowed. No scrap paper is allowed.
Make sure that there is nothing on your desk except pens and erasers.
If you need extra space, use the back of the page and indicate that you have done so.
Do not remove any page from the original staple. Otherwise, there will be 3 points off.
Show your work on your answer sheet if asked. If you do not show your work when asked, we will
give zero point for the problem even if your answer is correct.
We will not select among several answers. Make sure it is clear what part of your work you want
graded. If two answers are given, zero point will be given for the problem.
Throughout, you will receive full credit if someone with no understanding of probability, set theory,
and calculus could simplify your answer to obtain the correct numerical solution. However, you must
give a numerical answer where asked.
Problem 1 (a),(b)
Problem 2 (a),(b),(c),(d),(e)
Problem 3 (a),(b),(c)
Problem 4 (a),(b),(c),(d)

1. (7 points) Suppose that whether or not it rains today depends on previous weather conditions through
the last three days.
(a) (3 points) Define state Xn so that X = {Xn : n = 0, 1, . . .} becomes a Markov chain.

(b) (4 points) Find state space S.

2. (18 points) Let X = {Xn : n = 0, 1, 2, . . .} be a discrete time Markov chain on state space S = {0, 1, 2}
with initial distribution a(0) = (.2, .5, .3) and transition matrix

.4 .4 .2
P = .1 .4 .5 .
.5 .4 .1
IMPORTANT: There is no partial point. Any calculation error or mistake (e.g., reading matrix elements wrong) will result in a zero point. You can leave your answer in a form including addition,
subtraction, multiplication and division such as 0.999(0.111)+0.333(0.222)
. But you must give a numerical
0.9(0.1)+0.1(0.1)
answer in (d).
(a) (3 points) Find Pr{X2 = 1 | X0 = 1}.

(b) (3 points) Find Pr{X8 = 1, X9 = 2 | X6 = 1}.

(c) (4 points) Find a(1) . A numerical answer is expected.

(d) (5 points) Find Pr{X1 = 2 | X0 = X2 = 1}.

(e) (3 points) Find E[X12 ].

3. (10 points) A store opens from Monday to Friday. Suppose that the weekly demand D of a nonperishable product is iid with the following distribution:
d
Pr{D = d}

5
0.2

10
0.3

20
0.5

Unused items from one week can be used in the following week. The management decides to use the
following inventory policy: whenever the inventory level in Friday evening is less than or equal to 5,
an order is made and it will arrive by Monday morning. The order-up-to quantity is set to be 20. We
start with initial inventory 20.
Let Xn be the number of items in stock at the beginning of the nth week. It is known that X = {Xn :
n = 0, 1, . . .} is a discrete time Markov chain.
(a) (3 points) Give its state space.

(b) (2 points) Give its initial distribution.

(c) (5 points) Give the transition matrix.

4. (15 points) Suppose that coin 1 has probability .7 of coming up heads, and coin 2 has probability .6
of coming up heads. If the coin flipped today comes up heads, then we select coin 1 to flip tomorrow,
and if it comes up tails, then we select coin 2 to flip tomorrow. The coin initially flipped is coin 1 with
probability 0.9. Let Xn represent the index of a coin flipped on day n.
(a) (2 points) Find state space S.

(b) (2 points) Find initial distribution a.

(c) (4 points) Find the transition matrix P.

(d) (7 points) If we play this game for 1000 days, how many days do we expect to flip coin 1? A
numerical answer is expected. Also show your work on the answer sheet.