1. Formula definition
at time ttt observesxt x_{t}xt, then get TTT independent random variables( x 1 , . . . , x T ) − p ( X ) (x_{1},...,x_{T})-p(X)(x1,...,xT)−p(X)
From the conditional probability formula:
p ( a , b ) = p ( a ) p ( b ∣ a ) = p ( b ) p ( a ∣ b ) p(a,b)=p(a)p(b|a)=p(b)p(a|b) p(a,b)=p(a)p(b∣a)=p(b)p(a∣b)
The statistical formula of the sequence model can be obtained:
p ( X ) = p ( x 1 ) ⋅ p ( x 2 ∣ x 1 ) ⋅ . . . p ( x T ∣ x 1 , . . . , x T − 1 ) p(X)=p(x_{1})·p(x_{2}|x_{1})·...p(x_{T}|x_{1},...,x_{T-1}) p(X)=p(x1)⋅p(x2∣x1)⋅...p(xT∣x1,...,xT−1)
2. Sequence Modeling
The task of the sequence model can be seen as solving p ( xt ∣ x 1 , . . . , xt − 1 ) p(x_{t}|x_{1},...,x_{t-1})p(xt∣x1,...,xt−1) , which can be solved by modeling the conditional probability, namely:
p ( x t ∣ x 1 , . . . , x t − 1 ) = p ( x t ∣ f ( x 1 , . . . , x t − 1 ) ) p(x_{t}|x_{1},...,x_{t-1})=p(x_{t}|f(x_{1},...,x_{t-1})) p(xt∣x1,...,xt−1)=p(xt∣f(x1,...,xt−1))
其中 p ( x t ∣ x 1 , . . . , x t − 1 ) p(x_{t}|x_{1},...,x_{t-1}) p(xt∣x1,...,xt−1) means: before the givent − 1 t-1t−Under the premise of 1 data, find thettthThe probability of t data. p ( xt ∣ f ( x 1 , . . . , xt − 1 ) ) p(x_{t}|f(x_{1},...,x_{t-1}))p(xt∣f(x1,...,xt−1)) means: for the existingt − 1 t-1t−1 data to build a model, use this model to predict thettt data, also become an autoregressive model.
3. Modeling method
3.1 Markov method
Suppose the current data is only related to τ ττ past data, then:
p ( x t ∣ x 1 , . . . , x t − 1 ) = p ( x t ∣ f ( x t − τ , . . . , x t − 1 ) ) = p ( x t ∣ f ( x t − τ , . . . , x t − 1 ) ) p(x_{t}|x_{1},...,x_{t-1})=p(x_{t}|f(x_{t-τ},...,x_{t-1}))=p(x_{t}|f(x_{t-τ},...,x_{t-1})) p(xt∣x1,...,xt−1)=p(xt∣f(xt − τ,...,xt−1))=p(xt∣f(xt − τ,...,xt−1))
3.2 Latent variable method
Introduce latent variable ht h_{t}htTo represent past information ht = f ( x 1 , . . . , xt − 1 ) h_{t}=f(x_{1},...,x_{t-1})ht=f(x1,...,xt−1)，则 x t = p ( x t ∣ h t ) x_{t}=p(x_{t}|h_{t}) xt=p(xt∣ht)
Four. Summary
In a temporal model, current data is related to previously observed data;
In an autoregressive model, you use your own past data to predict future data.