Fall 2010

IE 361
HOMEWORK 1
Due: November 5,2010
1. Aylin and Barış are playing series of games of Rock, Paper, Scissors. Recall that Scissors cuts
Paper covers Rock smashes Scissors.
Aylin’s Strategy: Choose with equal probability
a. Play Scissors
b. Repeat Barış’s last play

Barış’s Strategy: Choose with equal probability

a. Repeat his last play
b. Play whatever would have beaten Aylin’s last play.

a. Define an appropriate Markov chain to analyze the situation. Provide a clear state
definition, the state space and the transition probability matrix.
b. Does stationary distribution of the Markov chain exist? If yes, find the stationary
distribution; else explain why there is no stationary distribution.
c. What is Aylin’s expected win:loss:draw ratio in the long run?

2. A group of 3 processors is arranged an ordered list. When a job arrives, the first processor in
line attempts it; if it is unsuccessful, then the next in line tries it; if it is too unsuccessful, then
the next in line tries it. When the job is successfully processed or after all processors have
been unsuccessful, the job leaves the system. At this point we are allowed to reorder the
processors, and a new job appears. Suppose that we use the one-closer re-ordering rule,
which moves the processor that was successful one closer to the front of the line by
interchanging its position with the one in front of it. If all processors were unsuccessful (or if
the processor in the first position was successful), then the ordering remains the same.
Suppose that each time processor i attempts a job then, independently of anything else, it is
successful with probability pi.= 0.8i

a. Define an appropriate Markov chain to analyze this system. Provide a clear state
definition, the state space and the transition probability matrix.
b. Find the long run probabilities. What is the long run probability that processor 1 is closer
to the front of the line than processor 3?

3. Consider a periodic-review, (s, S) inventory system, where the inventory level is reviewed at
the end of each day. If it is strictly less than s (the reorder point), then an order is placed to
increase the inventory level to S (the order-up-to point). Total number of items demanded in
a day n, Dn has the following probability distribution:

1
 0.5 k=0

P( D=
n k=
) 0.3 k=1
0.2
 k=2

Note that no backlogging is allowed so that the demands that are not satisfied are lost.

Let s=1, S=4 and compute the following:

a. The long-run mean inventory level

b. The long-run fraction of demand that is lost during a day
c. The expected long-run average daily profit if the daily inventory holding cost per item is
k, the cost of a replenishment order is o, cost of lost demand for each item is m and the
selling price an item is v
d. The expected number of days between successive replenishments

4. Amateur tennis player Mert’s performance in a tennis game is correlated to his morale. If he
has won the past two games, then he will win the next game with 0.7 probability. If he has
lost the past two games, then he will win the next game with 0.3 probability. If he has lost the
last game but won the game before that, with 0.4 probability he will win the next game.
Finally, if he won the last game but lost the game before that, with 0.6 probability he will win
the next game. From the rules of tennis game, no game ends up in a draw. Consider that Mert
has won his last two games.

a. Define an appropriate Markov chain. Provide a clear state definition, the state space and
the transition probability matrix.
b. What is the probability of first future loss being followed by another loss?
c. Define L as the number of games played until the first loss. What is the probability mass
function of L?
d. Do the steady-state probabilities exist? If yes, find them; else explain why they do not
exist.
e. What is the probability that Mert will win his 2000th game, given that the results of the
games 2000 and 2001 are the same? Find an approximate solution and clearly explain any
assumptions made in this approximation.
f. Define D as the number of games played until the sequence “loss-loss”, i.e. two losses in
a row. Find E[D].
g. Mert has announced in a press conference that he will end his career if he loses 3 games
in a row. Find the number of games that Mert will play from now on.