-EM machine learning algorithm -GMM model notes

GMM i.e., Gaussian mixture model, according to the following EM model derived from theoretical formula GMM:

　　　　X is a random variable with a Gaussian distribution formed by mixing K, take the respective Gaussian probability distribution is [Phi] _{. 1,} [Phi] _2, ..., [Phi] _K , the i-th mean Gaussian distribution is [mu] _i , [Sigma variance _i . If the observed series of samples of the random variable X X _{. 1} , X ₂ , ..., X _n- , try to estimate the parameters φ, μ, Σ.

　　　　E-step

　　　　

 M-step

　　　　The number and distribution parameters of the Gaussian distribution into the EM model:

　　　　

　　　　The partial derivative of the mean:

　　　　 

　　　　So that the above formula is equal to 0, the mean Solutions:

　　　　

　　　　The variance of the Gaussian distribution: the partial derivative equal to 0:

　　　　

　　　　Distribution of a number of parameters:

　　　　

　　　　get

　　　　

　　　　　　　　Lagrange multiplier method:

　　　　　　　　Since the probability distribution of the number is 1, the establishment of Lagrange equation:

　　　　　　　　

 

　　　　　　　　Solving φi non-negative constant, irrespective φi≥0 this condition, the partial derivatives equal to 0:

　　　　　　　　

So far GMM derivation formally completed.