Maximum possible marks: 30 marks

1. A Markov chain (Xn, n = 0, 1, 2, . . .) with state space S = {1, 2, . . . , N} has transition probabilities described as follows. For each pair of states i and j, there is a posi-tive number aij > 0. The number aij does not depend on the order of the pair i and j, in the sense thataji = aij       for all i, j S.

Set

and defifine the transition probabilities pij so that, for each fifixed starting state i, the transition probabilities pij , j S are proportional to aij :

For instance, if there are N = 3 states, then d2 = a21 + a22 + a23 and the transition matrix is

Show that there is an equilibrium distribution π = (π1, . . . , πN ) with the property that πi is proportional to di.          [8 marks]

1. A service facility has two streams of customers arriving at it. Stream A customers arrive as a Poisson process at rate 5 per hour. Stream B customers arrive as a Poisson process at rate 3 per hour. The two arrival streams are independent of each other. Suppose there were 5 arrivals at the facility between 9 a.m. and 11 a.m.

(a) What is the expected number of customers to arrive at the facility between 11.00 a.m. and 11.30 a.m.?

(b) What is the probability that the time between two successive arrivals from stream A is greater than 30 minutes?

(c) What is the probability that exactly 3 customers from stream A arrive at the service desk between 11.00 and 11.30?

(d) What is the probability that of the fifirst 4 customers to arrive at the facility after 11.00 a.m., 3 are from stream A, and 1 is from stream B?

(e) Suppose 5 customers arrived at the facility between 11 a.m. and 12 noon. What is the probability that 2 of those 5 arrivals occurred between 11 a.m. and 11.30 a.m.?           [10 marks]

1. Consider the Ehrenfest urn model with state space S = {0, 1, 2, 3, . . . , N} and 1-step transition matrix P =（pij）where transition probabilities are

(a) In Tutorial 4, function “fifillmat” produces the 1-step transition matrix P for the Gambler’s ruin process. Customize that code to produce the 1-step transition matrix P of the Ehrenfest urn model.

(b) Use part (a) and the built-in function “solve” in R to write a piece of code that solves

(Pmin I) × m[A] = 1,

where N = 6 and A = {0, 6} to fifind the values of miA, the expected times to reach the set A, starting from a state i for i = 1, 2, 3, 4, 5 (see Lecture 7 and Tutorial 4).

(c) Find matrix P 100 and matrix P 101 (see Tutorial 3 or Tutorial 4).

(d) In Lecture 6 we found the equilibrium distribution of the Ehrenfest urn model. Using Part(c) answer the following questions: