Machine Learning: Markov Models

It will be added in the future when suitable cases are encountered.

1 Markov model

  Markov Model (MM) is a statistical model widely used in natural language processing and other fields.

1.1 Mathematical definition

  Consider a set of random variable sequences X = { X 0 , X 1 , … , X t , … } X=\{X_{0},X_{1},\dots,X_{t},\dots\}X={ X0​,X1​,…,Xt​,…} , where$X_t$represents time $t$ random variables, and each random variable$X_t$The same set of values ​​is called the state space $S$。$S$ can be discrete or continuous.
  Suppose at time$0$ random variable$X_0$Follow the probability distribution $P(X_0)=\pi \left(0\right)$ is the initial state distribution. If a certain moment$t\ge 1$ random variable$X_t$With the random variable X t − 1 X_{t-1} at the previous moment$X_{t-1}$Between the conditional distribution $F(X_t|X_{t-1})$ , and$X_t$only depends on $X_{t-1}$, independent of past random variables $(X_0,X_1,\dots ,X_{t-2})$ , then$X$ has the Markov property and is called a Markov chain. That is,$$P(X_t|X_0,X_1,\dots ,X_{t-1})=P(X_t|X_{t-1}),t=1,2,\dots$$其中，$P(X_t|X_{t-1})$ is called the transition probability distribution of the Markov chain.
  In addition, if the conditional transition probability distribution and time$t$ is independent, it is called a time homogeneous Markov chain. That is,$$P(X_{t+s}|X_{t+s-1})=P(X_t|X_{t+1})$$   If a certain moment$t\ge 1$ random variable$X_t$with ex $n$ states are related, it is called$nth$ order Markov chain. That is,$$P(X_t|X_0\dots X_{t-1})=P(X_t|X_{t-n}{X}_{t-n+1}\dots X_{t-1})$$

  In addition to Markov properties, Markov chains may also have irreducible, recurrent, periodic and ergodic properties.

1.2 Two kinds of Markov chains

1.2.1 Discrete Markov chain

  If the above random variable $X_t(t=0,1,2,\dots ,)$ is defined in the discrete space$In\; S$ , it is called a discrete Markov chain, and its transition probability distribution can be expressed by a matrix. IfS = { 1 , 2 , … , n } S=\{1,2,\dots,n\}S={ 1,2,…,n } then the transition probability distribution matrix is:$$P=\left[\begin{array}{cccc}{p}_{11}& p_{12}& \dots & p_{1n}\\ p_{21}& p_{22}& \dots & p_2\\ ⋮& ⋮& \cdots & ⋮\\ p_n& p_n& \dots & p_{nn}\end{array}\right]\text{\hspace{1em}(1)}$$其中$p_{ij}=P(X_t=i|X_{t-1}=j)$ is a Markov chain at$t-1$ moment from state$j$ transfers to timettstate of$t$$probability\; of\; i$ . $p_{ij}\ge 0$且${\sum }_i{p}_{ij}=1$ .
  Markov chain at any timettThe state distribution of$t$$t-1$ 's state distribution and transition probability distribution, that is,$\pi \left(t\right)=P\pi (t-1)=P\cdot P\pi (t-2)$ . And so on$\pi \left(t\right)=P^t\pi \left(0\right)$

1.2.2 Continuous Markov chain

  If the state space $S$ is defined in a continuous space, then the sequence$X$ is called a continuous Markov chain. Then the transition probability distribution is represented by the probability transition kernel function. For any$x\in S,A\in S)$ , transition probability$$P(x,A)={\int }_{A}p(x,y)dy$$

References

1. "Statistical Learning Methods"