Consider again the Markov chain with the following transition probability graph:
This figure depicts a Markov chain with seven states. The possible states are 1, 2, up to 7. 1, 2, and 3 are depicted as a row of three circles, in this order, at the upper-left of the figures. 4, 5, 6,and 7 are depicted as four squares arranged in a square, with S_4 and S_5 to the right of S_3 (in this order), 6 directly below 4, and 7 directly below 5. In this figure, transitions are depicted by arrows from a circle (representing the source) to a circle (representing the destination). The source and the destination may be the same, and transition arrows may be labelled with a number or a variable indicating the probability that if we start at the source, we will be at the given destination at the next step. In the row consisting of 1, 2, and 3, there are self-loops within 1 and 3, which are circular arrows with both the tail and the head at the same state. These self-loops are labelled with probabilities of 0.4 and 0.6, respectively. There are also transitions between 1 and 2, as well as between 2 and 3; the probability from 1 to 2 is 0.6 and from 2 to 1 is 0.2, while the probability from 2 to 3 is 0.8 and from 3 to 2 is 0.4. Connecting the two components (the line with three states and the square with four states), there's only one transition from 4 to 3 with probability 0.4. Within the square, there's a transition from 4 to 5 with probability 0.3 and 4 to 6 with probability 0.3. From state 5, the only transition is to state 6, with probability 1; similarly, from state 7, the only transition is to state 5 with probability 1. Lastly, from state 6, there's a transition to state 4 with probability 0.2 and to state 7 with probability 0.8.
Find the steady state distribution of the Markov chain.
\pi _1=\,
unanswered
\pi _2=\,
unanswered
\pi _3=\,
unanswered
\pi _4=\,
unanswered
\pi _5=\,
unanswered
\pi _6=\,
unanswered
\pi _7=\,
To find the steady state distribution of the Markov chain, we need to solve the system of equations given by:
π_1 = π_1 * 0.4 + π_3 * 0.2
π_2 = π_1 * 0.6 + π_2 * 0.8
π_3 = π_2 * 0.4 + π_3 * 0.6
π_4 = π_4 * 0.3 + π_6 * 0.2
π_5 = π_4 * 0.3 + π_7 * 1
π_6 = π_5 * 1 + π_6 * 0.8
π_7 = π_7 * 1 + π_5 * 0.3
Now, we can solve this system of equations to find the steady state distribution.
To find the steady-state distribution of the Markov chain, we need to solve the system of equations:
π_1 = 0.6π_2 + 0.4π_4
π_2 = 0.2π_1 + 0.8π_3
π_3 = 0.4π_2
π_4 = 0.3π_4 + 0.2π_6
π_5 = π_7
π_6 = 0.3π_4 + 0.8π_7
π_7 = 0.2π_6
We can rewrite these equations in matrix form as:
|π_1| |0 0.6 0 0 0 0 0 | |π_1|
|π_2| = |0.2 0 0.8 0 0 0 0 | |π_2|
|π_3| |0 0.4 0 0 0 0 0 | |π_3|
|π_4| |0 0 0 1 0 0.2 0 | |π_4|
|π_5| |0 0 0 0 0 1 0 | |π_5|
|π_6| |0 0 0 0.3 0 0.8 0 | |π_6|
|π_7| |0 0 0 0 0 0 0 | |π_7|
We can solve this matrix equation by finding the eigenvector corresponding to the eigenvalue 1. However, since the matrix has a row of zeros, this can be simplified. From the equations, we can see that π_5 and π_7 are both zero, and π_1, π_4, and π_6 only depend on themselves. Therefore, we have:
π_5 = 0
π_7 = 0
π_1 = 0.6π_2 + 0.4π_4
π_4 = 0.3π_4 + 0.2π_6
π_6 = 0.3π_4 + 0.8π_7
Simplifying further, we have:
π_1 = 0.6π_2 + 0.4π_4
π_4 = 0.3π_4 + 0.2π_6
π_6 = 0.3π_4
We can solve these equations to find the steady-state distribution:
π_4 = π_4(0.3 + 0.2 + 0.3)
π_4 = π_4(0.8)
π_4 = 0.8π_4
Since the coefficient for π_4 on both sides of the equation is the same, we can set π_4 equal to any value. Let's set π_4 = 1. Then, we have:
π_4 = 1
π_6 = 0.3π_4 = 0.3(1) = 0.3
π_1 = 0.6π_2 + 0.4π_4 = 0.6π_2 + 0.4(1) = 0.6π_2 + 0.4
π_2 = 0.2π_1 + 0.8π_3 = 0.2(0.6π_2 + 0.4) + 0.8π_3 = 0.12π_2 + 0.08 + 0.8π_3
Simplifying further, we have:
0.88π_2 = -0.08 + 0.8π_3
0.88π_2 - 0.8π_3 = -0.08
22π_2 - 20π_3 = -2
Now we have two equations and two unknowns. We can solve this system of equations to find the values of π_2 and π_3.
22π_2 - 20π_3 = -2
π_2 + π_3 = 1
Multiplying the second equation by 20, we get:
20π_2 + 20π_3 = 20
Adding the two equations together, we have:
42π_2 = 18
π_2 = 18/42 = 3/7
Substituting this value into the second equation, we find:
π_3 = 1 - π_2 = 1 - 3/7 = 4/7
Therefore, the steady-state distribution of the Markov chain is:
π_1 = 0.6π_2 + 0.4π_4 = 0.6(3/7) + 0.4(1) = 18/35 + 2/5 = 0.857
π_2 = 3/7
π_3 = 4/7
π_4 = 1
π_5 = 0
π_6 = 0.3
π_7 = 0