In machine learning algorithms, Markov chain (Markov chain) is a very important concept. Markov chain, also known as discrete-time Markov chain, is named after Russian mathematician Andrei Markov (Russian: Андрей Андреевич Марков).
A Markov chain is a random process of transition from one state to another in the state space.
This process requires the property of "no memory":
the probability distribution of the next state can only be determined by the current state, and the events before it in the time series have nothing to do with it. This particular type of "memorylessness" is called the Markov property.
Markov chains have many applications as statistical models of real processes.
At each step of the Markov chain, the system can change from one state to another according to the probability distribution, and can also maintain the current state.
A change of state is called a transition, and the probabilities associated with different state changes are called transition probabilities.
The mathematical representation of a Markov chain is:
Since the probability of state transition at a certain moment only depends on the previous state, then only the transition probability between any two states in the system is required, and the model of this Markov chain is determined.
A Markov chain is
a random process of transition from one state to another in the state space.
This process requires the property of "no memory":
the probability distribution of the next state can only be determined by the current state, and the events before it in the time series have nothing to do with it.
Hidden Markov Model (HMM) is a statistical model used to describe a Markov process with hidden unknown parameters.
The difficulty is to determine the implicit parameters of the process from the observable parameters. These parameters are then used for further analysis, such as pattern recognition.