0. Preface
Markov chain is used as a tool to describe discrete information sources in Shannon communication , so a complete understanding of Markov chain is the prerequisite for understanding Shannon’s information theory
1. Definition of Markov chain
annotation:
The definition does not specify the capacity of the state space, and n is the length of the sequence.
Type A means that the conditional probability of n+1 and n at two adjacent moments is only related to the probabilities of these two steps, and has nothing to do with the previous probabilities. This is called no aftereffect . No aftereffect we can understand that the state at the previous moment has no aftereffect for the current state.
We use the state and the number of steps (/sequence label) to use a coordinate, (n,i), that is, the i-th state of the nth step . Therefore, for a random sequence X(n), each step has each state A certain probability.
Homogeneous Markov chain:
For the homogeneous Markov chain, remember , then the matrix P is the first transition matrix of the homogeneous Markov chain .
2. CK equation
Probability of m-step transition:
CK equation:
The meaning of the CK equation is that for the transition probability from (0,i) to (m+r,j), its value is equal to from (0,i) to (m,k), and then from (m,k) to ( The sum of the probabilities of all paths of m+r, j), where k is all possible values in the state space.
The theorem can be proved by drawing by yourself, and the formula to prove it is given below
For a homogeneous Markov chain, the transition probability has nothing to do with time/number of steps. The above formula is for an element ij, and the CK equation is written in matrix form, which can be written in the following form:
