Generate model-related algorithms: EM algorithm steps and formula derivation

introduction

The EM algorithm is a selection algorithm, which was proposed by Dempster et al. in 1977. It is used for the maximum likelihood estimation of the probability model parameters containing hidden variables (hidden variable), or the maximum posterior probability estimation for each selection of the EM algorithm. The generation consists of two steps: E step, expectation (expectation); M step, maximization (maximization), so this algorithm is called expectation maximization algorithm (expectation maximization algorithm), referred to as EM algorithm.

EM Algorithm Examples and Solutions

Three-coin model: Suppose there are 3 coins, respectively $A,B,and\; C$ are the probabilities of heads of these coins being$pp$ _$p$ and$q$ Carry out the following coin trial: first coin$A$ , select coin$B$ or coin$C$ , pick coin$B$ , choose coin$C$ ; Then toss the selected coin, the result of tossing the coin, if there is a head, it will be recorded as 1, and if there is a negative, it will be recorded as 0; $n $ experiments are repeated independently (here, n=10), and the observation results are as follows: 1 , 1
$$1,1,0,1,0,0,1,0,1,1$$
Assume that only the result of the coin toss can be observed, but the process of the coin toss cannot be observed. Ask how to estimate the probability of three-coin heads, the parameters of the three-coin model.
The model expression is:
$$\begin{array}{rl}P(y|\theta )& =\sum _{z}P(and,z|\theta )=\sum _{z}P(z|\theta )P(y|z,i)\\ & =\pi p^y(1-p{)}^y+(1-\pi \left)q^y\right(1-q{)}^{1-y}\end{array}$$
Here, the random variable $y$ is an observed variable, indicating that the result of a trial observation is 1 or 0; a random variable is a hidden variable, indicating that the unobserved coin toss$A$ snowflake; $i=(p,p,q)$ is the model parameter. This model is the generative model of the above data. Note that the random variable$The\; data\; of\; y$ can be observed, the random variable$The\; data\; for\; z$ are unobservable.
Express the observed data as$Y=(Y_1,Y_1,...,Y_1{)}^T$ , unobserved data is expressed as$Z=(Z_1,Z_2,...,Z_n{)}^T$ , then the likelihood function of the observed data is
$$P(Y|\theta )=\sum _{z}P(Z|\theta )P(Y|Z,\theta )$$
即
$$P(Y|\theta )=\prod _{j=1}^n[\pi p^{y_j}(1-p{)}^{y_j}+(1-\pi \left)q^{y_j}\right(1-q{)}^{1-y_j}]$$
Consider finding model parameters$i=(p,p,q)$ Specifies the same value, whereθ
$$\hat{i}=arg\underset{i}{\max }The\; problem\; of\; logP(Y|\theta )$$
has no analytical solution, and can only be solved by an iterative method. The EM algorithm is an iterative algorithm that can be used to solve this problem. The EM algorithm for the above problems is given below, and its derivation process is omitted.
The EM algorithm first selects the initial value of the parameter, which is recorded as$i^{(0)}=(p^{(0)},p^{\left(0\right)},q^{\left(0\right)})$, and then pass through the estimated values ​​of the following iteration parameters until convergence. SectionIIThe estimated value of the i iteration parameter is$\theta$$i^{\left(0\right)}=(p^{(i)},p^{\left(i\right)},q^{\left(i\right)})$. The i + 1 i+1of the EM algorithm$i+1$ iteration is as follows.
Step E: Calculate the model parameters${Pi}^{(i)},p^{\left(i\right)},q^{\left(i\right)}$ Observational data$y_i$Determine the function B of the function
$$m^{i+1}=\frac{\pi \left(p^{\left(i\right)}{)}^{y_j}\right(1-(p^{\left(i\right)}){)}^{y_j}}{\pi (p^{(i)}{)}^{y_j}(1-(p^{(i)}){)}^{y_j}+(1-\pi )(q^{(i)}{)}^{y_j}(1-(q^{(i)}){)}^{1-y_j}}$$
Step M: Compute new estimates of model parameters
$${Pi}^{(i+1)}=\frac{1}{n}\sum _{j=1}^n{m}_j^{(i+1)}$$
$$p^{(i+1)}=\frac{{\sum }_{j=1}^n{m}_j^{(i+1)}{y}_j}{{\sum }_{j=1}^n{m}_j^{(i+1)}}$$
$$q^{(i+1)}=\frac{{\sum }_{j=1}^n(1-m_j^{(i+1)})y_j}{{\sum }_{j=1}^n(1-m_j^{(i+1)})}$$
Perform numerical calculations, assuming that the initial value of the model parameters is
$${Pi}^{(0)}=0.5,p^{\left(0\right)}=0.5,q^{\left(0\right)}=0.5$$
对$y_j=1$与$y_j=0$均有$m_j^{(1)}=0.5$ .
According to the M-step calculation,
$${Pi}^{(0)}=0.5,p^{\left(0\right)}=0.6,q^{\left(0\right)}=0.6$$
According to the E step,
$$m_j^{(2)}=0.5,j=1,2,...,10Continue$$
to iterate and get
$${Pi}^{\left(0\right)}=0.5,p^{\left(0\right)}=0.6,q^{\left(0\right)}=0.6$$
Then get the model parameterθ \thetaMaximum likelihood estimation of$\theta$
$$\widehat{Pi}=0.5,\hat{p}=0.6,\hat{q}=0.6$$
$Pi=0.5$ means that coin A is well-balanced, and this result is easy to understand.
If the initial value${Pi}^{(0)}=0.4,p^{\left(0\right)}=0.6,q^{\left(0\right)}=0.7$ , then the maximum likelihood estimation of the model parameters obtained is$\widehat{Pi}=0.4064,\hat{p}=0.5368,\hat{q}=0.6432$ . That is to say, the EM algorithm is related to the selection of initial values, and choosing different initial values ​​may result in different parameter estimates.

EM Algorithm Steps and Description

Generally, Y is used to represent the data of observed random variables, and Z is used to represent the data of hidden random variables. The combination of Y and Z is called complete-data, and the observed data Y is also called incomplete-data. Suppose given observation data Y, its probability distribution is $P(Y|\theta )$ , where is the model parameter to be estimated, then the likelihood function of incomplete data Y is$P(Y|\theta )$ , log likelihood function$L(0)=logP(Y|\theta )$ ; Suppose the joint probability distribution of Y and Z is$P(YZ|\theta )$ , the complete log likelihood function is$logP(Y,Z|\theta )$
LetL ( θ ) = log P ( Y ∣ θ ) L(\theta)=logP(Y|\theta).$L\left(i\right)=$The maximum likelihood estimation of$l$$o$$g$$P$$($$Y$$|$$\theta$$)\; contains\; two\; steps\; in\; each\; generation:\; E\; step,\; seeking\; expectation:\; M\; step,\; seeking\; maximization,\; and\; the\; EM\; algorithm\; is\; introduced\; below.$
Input: observed variable data$Y$ , hidden variable data$Z$ , joint distribution$P(Y,Z|\theta )$ , conditional distribution$P(Z|Y,\theta )$ ;
output: model parameters$\theta$ .
(1) Select the initial value of the parameter$i^{\left(0\right)}$ , start iteration;
(2) E-step: remember$i^{\left(i\right)}$ for$The\; estimated\; value\; of\; the\; parameter\; of\; the\; i$ generation, at the$i+For\; the\; 1st$ equation E, the function
$$\begin{array}{rl}Q(\theta ,i^{(i)})& =E_Z[logP(Y,Z|\theta )|Y,i^{(i)}]\\ & =\sum _{Z}logP(Y,Z|\theta )P(Z|Y,i^{(i)})\end{array}$$
For example, $P(Z|Y,\theta )$ in the given observation data$Y$ and the current parameter estimate$i^{\left(i\right)}$ Lower hidden variable dataZZThe conditional probability distribution of$Z$
; (3)M step: find$Q(\theta ,i^{\left(i\right)})$Maximized$\theta$ , determine$i+The\; estimated\; value\; of\; the\; parameter\; of\; 1$ iteration$i^{(i+1)}$
$$i^{(i+1)}=\arg \underset{i}{\max }Q(\theta ,i^{\left(i\right)})$$
(4) Repeat steps (2) and (3) until convergence.
Function$Q(\theta ,i^{\left(i\right)})$is the core of the EM algorithm, called the Q function (Q function).
The following are some explanations about the EM algorithm:
Step (1) The initial value of the parameter can be selected arbitrarily, but it should be noted that the EM algorithm is sensitive to the initial value Step (2) Step E to find$Q(\theta ,i^{\left(i\right)})$. In the Q function formula, Z is unobserved data, Y is observed data Note,$Q(\theta ,i^{\left(i\right)})$The first variable represents the parameter to be maximized, and the second variable represents the current estimated value of the parameter. Each iteration is actually asking for$The\; Q$ function is extremely large.
Step (3) M steps to find$Q(\theta ,i^{\left(i\right)})$to maximize, get$i^{(i+1)}$ , completed first generation$i^{(i)}->i^{(i+1)}$ . It will be proved later that each iteration increases the likelihood function or reaches a local extremum.
Step (4) gives the condition to stop iteration, generally for smaller positive numbers${ϵ}_1,{ϵ}_2$, if it satisfies∣
$$||{\theta }^{(i+1)}-i^{(i)}||<{ϵ}_{1}or||Q({\theta }^{(i+1)},i^{(i)})-Q({\theta }^{(i)},i^{(i)})||<{ϵ}_2$$
then stop iterating