Exercise: Steady-state calculation
0.0/4.0 points (graded)
Consider again the Markov chain with the following transition probability graph:
This figure depicts a Markov chain with seven states. The possible states are 1, 2, up to 7. 1, 2, and 3 are depicted as a row of three circles, in this order, at the upper-left of the figures. 4, 5, 6,and 7 are depicted as four squares arranged in a square, with S_4 and S_5 to the right of S_3 (in this order), 6 directly below 4, and 7 directly below 5. In this figure, transitions are depicted by arrows from a circle (representing the source) to a circle (representing the destination). The source and the destination may be the same, and transition arrows may be labelled with a number or a variable indicating the probability that if we start at the source, we will be at the given destination at the next step. In the row consisting of 1, 2, and 3, there are self-loops within 1 and 3, which are circular arrows with both the tail and the head at the same state. These self-loops are labelled with probabilities of 0.4 and 0.6, respectively. There are also transitions between 1 and 2, as well as between 2 and 3; the probability from 1 to 2 is 0.6 and from 2 to 1 is 0.2, while the probability from 2 to 3 is 0.8 and from 3 to 2 is 0.4. Connecting the two components (the line with three states and the square with four states), there's only one transition from 4 to 3 with probability 0.4. Within the square, there's a transition from 4 to 5 with probability 0.3 and 4 to 6 with probability 0.3. From state 5, the only transition is to state 6, with probability 1; similarly, from state 7, the only transition is to state 5 with probability 1. Lastly, from state 6, there's a transition to state 4 with probability 0.2 and to state 7 with probability 0.8.
Find the steady state distribution of the Markov chain.
\pi _1=\,
unanswered
\pi _2=\,
unanswered
\pi _3=\,
unanswered
\pi _4=\,
unanswered
\pi _5=\,
unanswered
\pi _6=\,
unanswered
\pi _7=\,
To find the steady-state distribution of the Markov chain, we need to solve the system of equations:
\pi_1 = \pi_1(0.4) + \pi_2(0.6) + \pi_4(0.3)
\pi_2 = \pi_1(0.6) + \pi_2(0.2) + \pi_3(0.4)
\pi_3 = \pi_2(0.8) + \pi_3(0.6)
\pi_4 = \pi_1(0.3) + \pi_4(0.2) + \pi_5(1)
\pi_5 = \pi_4(0.3) + \pi_7(1)
\pi_6 = \pi_4(0.3) + \pi_6(0.2) + \pi_7(0.8)
\pi_7 = \pi_5(1)
We also have the equation:
\pi_1 + \pi_2 + \pi_3 + \pi_4 + \pi_5 + \pi_6 + \pi_7 = 1
Solving this system of equations will give us the values of \pi_1, \pi_2, \pi_3, \pi_4, \pi_5, \pi_6, and \pi_7.
To find the steady-state distribution of the Markov chain, we need to solve the equation:
πP = π
Where π is the steady-state distribution and P is the transition probability matrix. In this case, the transition probability matrix is:
P =
0.6 0.4 0 0 0 0 0
0.2 0 0.8 0 0 0 0
0 0.6 0 0 0 0 0
0 0 0 0 0.3 0 0.7
0 0 0 0 0 1 0
0 0 0 0.2 0 0 0.8
0 0 0 0 1 0 0
To solve this equation, we can set up a system of linear equations:
π1 = 0.6π1 + 0.2π2
π2 = 0.4π1 + 0.6π2 + 0.6π3
π3 = 0.8π2
π4 = 0.3π4 + 0.2π6
π5 = 0.3π4 + π7
π6 = 0.7π4 + 0.8π6
π7 = 0.8π6
Simplifying these equations, we get:
0.4π1 - 0.2π2 = 0
-0.4π1 + 0.4π2 - 0.6π3 = 0
-0.8π2 + π3 = 0
-0.7π4 + 0.2π6 = 0
-0.3π4 + π5 - π7 = 0
-0.7π6 + 0.8π4 = 0
-0.8π6 + π7 = 0
Using a linear algebra calculator or a matrix solver, we can find the solution to this system of equations. The steady-state distribution of the Markov chain is:
π1 = 0.0
π2 = 0.0
π3 = 0.0
π4 = 0.0
π5 = 0.0
π6 = 0.0
π7 = 0.0