Transition Probability Matrix is the most important thing to know when studying stochastic processes because transition matrices represent the probabilities of how the systems can change. These tools are essential in presenting and examining the probability of changes from one state to another within a given system.
This article, thus, introduces Transition Probability Matrix with some practical applications and solved examples.
What is Transition Probability Matrix?
Markov's process principle of Transition Probability Matrix which is also noted as P is an important thing. A Markov chain is a kind of stochastic process, which fulfils the Markov property, i.e., the probability that the next state will occur only depends on the present state and not on the sequence of states which preceded it. By this property, Markov chains can be an effective and comfortable way of modelling random processes.
The Transition Probability Matrix consists of the transition probabilities from the initial state to the final state. Working with this matrix, if we set the matrix P as the Transition Probability Matrix then, the matrix elements Pij are the probability of transition from the ith state to jth state at a single step. This matrix is also the one that highlights the fact of the summation of the total probability of transition from one state to any other state and it equals 1, that is the row elements sum to 1.
Mathematical representation of Transition Probability Matrix
Let's think about a Markov chain with n states, named as 1,2,.....n. The transition probability matrix P is an n×n matrix where each element Pij gives the probability of moving from state i to state j in a one-time step. The matrix below illustrates the relationship between the Transitive Probability Matrix and P.
Properties of Transition Probability Matrix
The properties of Transition Probability Matrix, P can be given as follows:
- As a rule, not a single element of Transition Probability Matrix, P, should be found negative. The reason being that the negative probability is not acceptable as the very notion or it would lead to inconsistency with the definition of the notion of probability. It's just:
P_{ij} \geq 0 , for all i and j in the matrix.
- All the elements on a specific line of the Transition Probability Matrix should be positive for the matrix to be closer to 1. For example, on each of the rows of the matrix P, all of the components must add up to 1. In this way, the total probability from one of the states over all possible changes equals 1. Mathematically this could also be given like:
\sum_j P_{ij} = 1 for all i
- Transition probability indeed relates to the future state in different states has the property that the next state, that is, the state of the system after it goes through that transition depends only on the current state and is independent of any other prior states.
- Transition probability matrix attains the stationary distribution. Typically, a stationary distribution is the probability distribution that the matrix remains unchanged after applying the transitions defined by the matrix. If there is a stationary distribution π; then, πP = π. Therefore, that distribution is invariant and corresponds to the eigenvalue of 1.
Applications of Transition Probability Matrix
Transition Probability Matrix is used in various fields some of which are mentioned as follows:
- Economics: Transition matrices mimic the incoming innovations either into a displaced pattern or approximation trends in consumer consumption as well as business investment. In this way, economists can predict future states of the economy or the stock market.
- Genetics: In genetics, these matrices are of great help as they form the basis for modeling the probabilities of the inheritance of traits and the transitions of species over generations.
- Computer Science: Recommended algorithms particularly those for learning machines use transition matrices to evaluate decisions and advance predictive capacities over time.
Constructing a Transition Probability Matrix
Building a transition probability matrix involves defining the probabilities of the movement from one state to another in a Markov chain. Getting through this requires a clear understanding of the maximality as well as the dynamics of the process of passing from one state to another. Here is a step-by-step guide to constructing a Transition Probability Matrix:
Step 1: Defining the states
Determine ranges of all potential states of the system. Let’s denote the number of states as 'n'. The states themselves should be unique and cover all possible situations of the system.
Example: Consider a simple weather model with three states:
- Sunny (S)
- Cloudy (C)
- Rainy (R)
Step 2: Determining the Transition Probabilities
For each state, specify the probability of transition to each of the other states (including remaining at the initial state). These probabilities can be inferred from historical data, or estimated from theoretical models and expert judgement.
Example: Suppose we have the following transition probabilities based on historical weather data:
- From Sunny to Sunny: 0.6
- From Sunny to Cloudy: 0.3
- From Sunny to Rainy: 0.1
- From Cloudy to Sunny: 0.4
- From Cloudy to Cloudy: 0.4
- From Cloudy to Rainy: 0.2
- From Rainy to Sunny: 0.2
- From Rainy to Cloudy: 0.5
- From Rainy to Rainy: 0.3
Step 3: Constructing the Matrix
Design a square matrix where each row and column is a unique state. Supply the matrix with the transition probabilities determined in Step 2. Make the sum of each row equal to the value 1, which is the total probability of transitioning from the current state to any other state.
Step 4: Verification of the Matrix
Check that every line adds up to 1 and the probabilities are all non-negative.
Step 5: Interpret the Matrix and Use that Interpretation
Reading and applying the concepts of the specific matrix are two processes of visualizing & understanding data which are extremely simple tasks.
One could use the transition probability matrix to analyze the system's behavior over time. Using this matrix, one can predict future states, calculate steady-state distributions, and also understand the long-term behavior of the system.
Solved Problems on Transition Probability Matrix
Problem 1: Given below the Transition Probability Matrix (P)
Find the probability of passing from state 1 to state 3 in 1 step.
Solution:
Transition probability from state 1 to the state 3 in 1 step can be found from the entry P13.
∴ P13 = 0.6
So, the probability that this transition occurs is 0.6.
Problem 2: Given below the Transition Probability Matrix (P)
Find the stationary distribution π.
Solution:
Stationary distribution π should satisfy πP = π and
\sum_i \pi_i = 1 .Let π = (π1, π2)
∴ π1 = 0.5×π1 + 0.4×π2
∴ π2 = 0.5×π1 + 0.6×π2
Also π1 + π2 = 1, so
∴ π2 = 1 - π1
Substituting the value of π2, we get
∴ π1 = 0.5×π1 + 0.4×(1 - π1)
∴ π1 = 0.5×π1 + 0.4 - 0.4×π1
∴ π1 = 0.1×π1 + 0.4
∴ 0.9×π1 = 0.4
∴ π1 = 4/9
∴ π2 = 1 - 4/9 = 5/9
So, the stationary distribution is π = (4/9, 5/9)
Problem 3: Given below the Transition Probability Matrix (P).
Find the 2 step transition probability for the transition from state 1 to state 3.
Solution:
In order to find the 2 step transition probabilities, we need P2.
∴ P2 = P x P
On calculating the elements of P2:
\therefore P^2 = \begin{pmatrix} (0.1 \cdot 0.1 + 0.6 \cdot 0.4 + 0.3 \cdot 0.2) & (0.1 \cdot 0.6 + 0.6 \cdot 0.4 + 0.3 \cdot 0.5) & (0.1 \cdot 0.3 + 0.6 \cdot 0.2 + 0.3 \cdot 0.3) \\ (0.4 \cdot 0.1 + 0.4 \cdot 0.4 + 0.2 \cdot 0.2) & (0.4 \cdot 0.6 + 0.4 \cdot 0.4 + 0.2 \cdot 0.5) & (0.4 \cdot 0.3 + 0.4 \cdot 0.2 + 0.2 \cdot 0.3) \\ (0.2 \cdot 0.1 + 0.5 \cdot 0.4 + 0.3 \cdot 0.2) & (0.2 \cdot 0.6 + 0.5 \cdot 0.4 + 0.3 \cdot 0.5) & (0.2 \cdot 0.3 + 0.5 \cdot 0.2 + 0.3 \cdot 0.3) \end{pmatrix} Now, for the entry
P_{13}^{2} :
\therefore P_{13}^2 = 0.1 \cdot 0.3 + 0.6 \cdot 0.2 + 0.3 \cdot 0.3
\therefore P_{13}^2 = 0.03 + 0.12 + 0.09 = 0.24 Therefore, the probability that state 1 transits to state 3 in two steps is 0.24.
Problem 4: Given below the Transition Probability Matrix (P)
What are the absorbing states?
Solution:
An absorbing state is a state which means that the diagonal entry for that state is 1 whereas all the other entries in that row are 0. Now, from the given matrix, we get,
\therefore P = \begin{pmatrix} 1 & 0 & 0 \\ 0.2 & 0.8 & 0 \\ 0.3 & 0.4 & 0.3 \end{pmatrix} First row is (1, 0, 0) indicates that once the process enters state 1, it will never leave. Based on this, step 1 is considered as an absorbing state.
Problem 5: Given below the Transition Probability Matrix (P)
Find out whether it is irreducible or not.
Solution:
Generally a matrix is considered to be irreducible if one can reach from each and every state to all the other states in a finite number of transitions. Now, on analyzing the given matrix
- To go from state 1 to state 2 or state 3 one must make a transition
- State 2 can only remain in state 2.
- State 3 can move to state 1, 2 or can stay in state 3.
The matrix is not irreducible because the state 2 cannot make a transition to state 1 or 3.
Problem 6: Given below the Transition Probability Matrix (P)
Find whether the matrix is aperiodic or not.
Solution:
The period of a state i, is the GCD of the lengths of all paths which take back to i.
∴ P11 = 0.5
∴ P22 = 0.5
For the greatest common divisor of all path lengths and each state to be not less than 1, the given probability matrix is aperiodic.
Practice Questions on Transition Probability Matrix
Problem 1: Given the Transition Probability Matrix P:
Find the stationary distribution π.
Problem 2: Given the Transition Probability Matrix P
Find the 3 step transition probability for the transition from state 3 to state 1.
Problem 3: Given the Transition Probability Matrix P.
Find out whether it is aperiodic or not and determine the period of each state.
Problem 4: Given the Transition Probability Matrix P:
Find the steady state distribution π and verify its correctness.
Problem 5: Given the Transition Probability Matrix P:
Find any absorbing states if any, and determine whether the matrix is aperiodic or not.
Problem 6: Given the Transition Probability Matrix P:
Find the 5-step transition probability for the transition from state 1 to state 2.
Problem 7: Given the Transition Probability Matrix P:
Is this given Markov chain the ergodic one or not?
Hint: Wielandt's theorem says that a Markov chain becomes ergodic if and only if every element of this matrix Pm where m = (n-1)2 + 1 of the matrix is positive, where P is the Transition Probability Matrix. At the same time, n is the number of states of the matrix.
Conclusion
The transition probability matrix P is a powerful way to denote the dynamics of a Markov chain. Of course, this avenue is efficient when properly utilized and this is not the exception with the transition probability matrix. The matrix P is the instrument that gets the randomness of the values to a place where it can be analyzed properly. This matrix is used to numerically describe the relationship among different states of Markov Chain and it is applied to proposing a generalizing specification of the Markov Chain.
Read More: