Getting to Know the Markov Model (Markov Model)

1. Concept

A Markov Model is a probabilistic model used to describe a time-varying probability distribution in a stochastic system. Markov models are based on the Markov assumption that the current state is only related to its previous state and has nothing to do with other states.

Two, nature

The Markov model has the following properties:

① Markov property : that is, the next state of the Markov model is only related to the current state and has nothing to do with the historical state.

② Normalization : The sum of all state transition probabilities is 1, that is, for any state i, there is ${\sum }_{j}p(i,j)=1$。

③ No aftereffect : The state transition of the Markov model has no aftereffect, that is, the probability distribution starting from a certain state is not affected by the previous state.

④ Stability : The state transition probability of the Markov model is fixed and time invariant.

These properties make the Markov model widely used in many fields such as statistics, economics, and computer science, and play an important role.

3. Learning steps

To learn the Markov model, you can follow the steps below:

① Understand the concept and basic definition of Markov model, including Markov property, normalization, no aftereffect and stability.

② Learn the basic principles of the Markov model, including concepts such as state transition probability, transition matrix, and Markov chain.

③ Learn how to build a Markov model through examples, and understand how to use the Markov model to solve practical problems. For example, if you want to learn about weather forecasting, you can build a Markov model where states represent the weather (sunny, cloudy, rainy) and state transition probabilities represent transitions in the weather.

④ Learn the application of Markov model, such as text generation, recommendation system, speech recognition, etc.

⑤Practice writing codes to gain a deep understanding of the implementation details of the Markov model.

import numpy as np

def markov_model(states, transition_prob):
    current_state = states[0]
    while True:
        print(current_state)
        index = states.index(current_state)
        next_index = np.random.choice(len(states), p=transition_prob[index])
        current_state = states[next_index]

# 创建状态列表
states = ["晴天", "阴天", "雨天"]

# 创建转移概率矩阵
transition_prob = [[0.8, 0.2, 0.0], [0.6, 0.3, 0.1], [0.2, 0.5, 0.3]]

# 运行模型
markov_model(states, transition_prob)

transition_prob is a transition probability matrix, which represents the transition probability between different states. Each row represents the transition probability of a state, and each column represents the probability of a state.

For example, the first row [0.8, 0.2, 0.0] means that the probabilities of transitioning from the "sunny" state to "sunny", "cloudy", and "rainy" are 0.8, 0.2, and 0, respectively. Therefore, during the model run, if the current state is "sunny", then it has an 80% probability of continuing to remain "sunny", a 20% probability of shifting to "cloudy", and a 0% probability of shifting to "rainy".

Therefore, transition_prob allows us to describe the random transition of states, and in the Markov model, it is the basis for simulating state transitions.