EM algorithm to solve three-coin model parameter derivation

EM algorithm

The EM algorithm is an iterative algorithm used for parameter estimation of probabilistic models containing latent variables. The algorithm alternates between two steps: E-step (Expectation Step, expectation step) and M-step (Maximization Step, maximization step). In step E, the current parameter estimates are used to calculate the posterior probability of the hidden variable; in step M, the posterior probability of the hidden variable calculated in step E is used to maximize the log-likelihood function of the complete data to obtain a new parameter estimates. These two steps are iterated until convergence.

EM algorithm general process

The EM algorithm finds $L\left(i\right)=\log P(Y\mid$Maximum likelihood estimate of$\theta$$)\; .$Each iteration contains two steps,$Step\; E$ , find expectation,$M$ step, seek to maximize.

Input: observed variable data $Y$ , hidden variable data$Z$ , joint distribution$P(Y,Z|\theta )$ , constant$P(Z|Y,\theta )$ ;
Target output: model parameter$i$ .

The following are the iteration steps:
(1) Select the initial value of the parameter $i^{\left(i\right)}$ , start iteration;
(2)$E$ -step: 记$i^{\left(i\right)}$ Part$i$ iteration parameter$The\; estimated\; value\; of\; \theta$ , at the$i+$E \mathrm{E}$for\; 1$ iteration$E$步, the function
$$\begin{array}{rl}Q\left(i,i^{(i)}\right)& =E_Z\left[\log P(Y,Z|i)|Y,i^{(i)}\right]\\ & =\sum _Z\log P(Y,Z|i)P\left(Z|Y,i^{(i)}\right)\end{array}$$

Specifically, $P\left(Z|Y,i^{\left(i\right)}\right)$is the given observation data$Y$ and the current parameter estimate$i^{\left(i\right)}$ Lower hidden variable data$Conditional\; probability\; distribution\; of\; Z$ ;
(3)$\mathrm{M}$ 步: 求使 $Q(i,i^{\left(i\right)}$ Maximumθ \$theta$$\theta$ , determine the$i+The\; estimated\; value\; of\; the\; parameter\; for\; 1$ iteration$i^{(i+1)}$
$$i^{(i+1)}=\arg \underset{i}{\max }Q\left(i,i^{\left(i\right)}\right)$$
(4) Useful$i^{(i+1)}$ is brought in, and steps (2) and (3) are repeated until convergence.

three coin model

First, I will introduce an example of using the EM algorithm (quoted from Professor Li Hang's "Statistical Learning Methods (2nd Edition, page 175)"). The book does not describe how the parameters of this example are obtained, so I have updated it here. A basic and detailed addition.

Example 9.1 ( Three-Coin Model ) Suppose there are 3 coins, denoted A, B, and C respectively. The probabilities of these coins appearing heads are $Pi,p$ and$q$ . Conduct the following coin tossing test: First toss coin$A$ , select coin$B$ or coin$C$ , choose coin$B$ , choose the coin$C$ ; Then toss the selected coin. The result of tossing the coin is recorded as 1 if it is heads and as 0 if it is tails; Repeat$n$ trials (here,$n=10$ ), the observation results are as follows:
$$1,1,0,1,0,0,1,0,1,1$$

Suppose we can only observe the result of coin tossing, but not the process of coin tossing, that is, we do not know the heads and tails of coin A in an experiment. Ask how to estimate the probability of heads of three coins based on a batch of experimental data, that is, the parameters of the three-coin model.

Express this model using mathematical formulas

$$\begin{array}{r}P(and\mid i)=\sum _{z}P(and,z|i)\\ =\sum _{z}P(z|i\left)P\right(y|z,i\end{array}$$

{ p ( y = 1 ∣ θ ) = π p + ( 1 − π ) q , solver p ( y = 0 ∣ θ ) = π ( 1 − p ) + ( 1 − π ) ( 1 − q ) , Let p ( y ∣ θ ) = π py ( 1 − p ) 1 − y + ( 1 − π ) yqy ( 1 − q ) 1 − y , y ∈ { 0 , 1 } \begin{aligned} & \left\{\begin{array}{l} p(y=1 \mid \theta)=\pi p+(1-\pi) q, \text { embedded range } \\ p(y=0 \mid \theta )=\pi ( 1 - p ) + ( 1 - \ pi ) ( 1 - q ) , \ text { enclosure field } \ end { array } \ right . \\ & given p(y \mid \theta)=\pi p^y(1-p)^{1-y}+(1-\pi)^yq^y(1-q)^{1-y }, and \in\{0.1\}\end{aligned}​{ p ( and=1∣i )=πp+(1−π ) q , Observed the front p ( and=0∣i )=p ( 1−p)+(1−p ) ( 1−q), observed  the oppositeSo p ( y∣i )=πpy(1−p)1−y+(1−Pi )yqy(1−q)1−y,y∈{ 0,1}​

In the above model $y$ indicates that the final observation result of an experiment is 1 or 0, and the random variable$z$ is a latent variable representing the unobserved coinAAThe result of$A$$i=(Pi,p,q)$ is the parameter of this model.

We write the observation data $P\; (\; Y$ ∣ θ ) = ∑ ZP ( Z ∣ θ ) P ( Y ∣ Z , θ ) P(Y \mid \theta )
$$P(Y\mid i)=\sum _{Z}P(Z|i\left)P\right(Y|Z,i)$$

$$P(Y|i)=\prod _{j=1}^n\left[\pi p^{y_j}(1-p{)}^{1-y_j}+(1-\pi \left)q^{y_j}\right(1-q{)}^{1-y_j}\right]$$

Our goal is to find the model parameters $i=(Pi,p,q)$ Standard value
$$\hat{i}=\underset{i}{\mathrm{argmax}}[\log P(Y|i)]$$

To solve this problem, we use $EM$ algorithm solves the above three-coin model

Review conditional expectations knowledge points

Conditional expectation refers to the calculation of the expectation of a random variable given certain conditions. Specifically, let $X$和$Y$ are two random variables,YYThe value range of$Y$$y_1,y_2,\dots ,y_n$, then given YYUnder the condition of$Y$ , XXConditional expectation of$X$$E(X|Y)$ is defined as:

$$E(X|Y)=\sum _{i=1}^n{X}_{i}P(X_i|Y)$$

Among them, $X_i$means $X$在$Y=y_i$The value when , $P(X_i|Y)$ expressed in$Y=y_i$Under the conditions, $X$ take$X_i$The probability. The meaning of this formula is that for each $y_i$，计算在$Y=y_i$Under the conditions, $The\; expected\; value\; of\; X$ , then for alliiSum$i$$X$ at givenYYExpected value under the conditions of$Y.$

EM algorithm

Here we focus on dismantling the formulas of the E step and the M step. The parameters θ \theta obtained after the M step are$\theta$ can proceed to the next round of iteration

E step

We introduce QQ in the above process$Q$ function (I won’t prove how it came from here, let’s use it first. For the detailed process of deriving the Q function, please see the reference article at the end of the article)

$$\begin{array}{rl}Q\left(i,i^{(i)}\right)& =E_Z\left[\log P(Y,Z\mid i)|Y,i^{(i)}\right]\\ & =\sum _Z\log P(Y,Z|i)P\left(Z|Y,i^{(i)}\right)\end{array}$$

Prototype $P\left(Z|Y,i^{\left(i\right)}\right)$According to different$z$ disassembly
for where$z=In\; the\; case\; of\; 1$ , use Bayesian formula to deduce ($A$ is the front, remember the hidden variable$z=1$ , means$y$ observations are from$Form\; B$ )
$$\begin{array}{rl}P(z=1|y,i^{\left(i\right)})=\frac{P(z=1,y,i^{\left(i\right)})}{P(and,i^{\left(i\right)})}& =\frac{P(z=1)P(y,i^{\left(i\right)}|z=1)}{{\sum }_{z}P(z)P(y,i^{(i)}|z)}\\ & =\frac{P(z=1)p(y,i^{\left(i\right)}|z=1)}{P(z=1)p(y,i^{\left(i\right)}|z=1)+P(z=0\left)p\right(y,i^{\left(i\right)}|z=0)}\\ & =\frac{\pi p^y(1-p{)}^{1-y}}{\pi p^y(1-p{)}^{1-y}+(1-\pi \left)q^y\right(1-q{)}^{1-y}}\end{array}$$
In the same way, $z=0$时
$$\begin{array}{rl}P(z=0\mid y,i^{(i)})=\frac{P(z=0,y,i^{(i)})}{P(and,i^{\left(i\right)})}& =\frac{P(z=0)P(y,i^{\left(i\right)}|z=0)}{{\sum }_{z}P(z)P(y,i^{(i)}|z)}\\ & =\frac{P(z=0)p(y,i^{\left(i\right)}|z=0)}{P(z=1)p(y,i^{\left(i\right)}|z=1)+P(z=0\left)p\right(y,i^{\left(i\right)}|z=0)}\\ & =\frac{(1-\pi )q^y(1-q{)}^{1-y}}{\pi p^y(1-p{)}^{1-y}+(1-\pi \left)q^y\right(1-q{)}^{1-y}}\end{array}$$
Definition $m^{\left(i\right)}=p(z=1|y,i^{\left(i\right)})$represents the probability that the y observation result comes from coin B (A is positive), then the probability that the coin comes from C (A result is negative) is$1-m^{\left(i\right)}$
At this time,$Q\left(i,i^{\left(i\right)}\right)$functional function
$$\begin{array}{rl}Q\left(i,i^{(i)}\right)& =E_Z\left[\log P(Y,Z\mid i)|Y,i^{(i)}\right]\\ & =\sum _Z\log P(Y,Z|i)P\left(Z|Y,i^{\left(i\right)}\right)\\ & =\log P(Y,Z=1|i)P\left(Z=1|Y,i^{\left(i\right)}\right)+\log P(Y,Z=0|i)P\left(Z=0|Y,i^{\left(i\right)}\right)\\ & =\log P(Z=1|i\left)P\right(Y|Z=1,i)\cdot m^{(i)}+\log P(Z=0|i\left)P\right(Y|Z=0,i)\cdot \left(1-m^{\left(i\right)}\right)\\ & =m^{\left(i\right)}\cdot \log \left[\pi p^y(1-p{)}^{1-y}\right]+\left(1-m^{\left(i\right)}\right)\cdot \log \left[(1-\pi )q^y(1-q{)}^{1-y}\right]\\ & =m^{\left(i\right)}\left[\log Pi+\log p^y(1-p{)}^{1-y}\right]+\left(1-m^{\left(i\right)}\right)\cdot \left[\log (1-Pi)+\log q^y(1-q{)}^{1-y}\right]\end{array}$$

M step

求使 $Q(i,i^{\left(i\right)}$ Maximumθ \$theta$$\theta$ , determine the$The\; estimated\; value\; of\; the\; parameter\; for\; i$ iteration$i^{(i+1)}$ . Let there be a total of$n$ group of observation values$y_j,1\le j\le n$ , then$The\; Q$ function can be expressed as$Q^{\prime }={\sum }_j^{n}Q\left(i,i^{\left(i\right)}\right)$. Find$Q^{\prime }$ Maximizing parameters$i=(Pi,p,q)$ , we let$Q^{}$The partial derivative of${}^{\prime }$$0$ , respectively find the parameters$i=(Pi,p,q)$。

The following shows how to solve for $Definitions$ :
$$\begin{array}{rl}\frac{\partial Q^{\prime }}{\partial \pi }=\sum _j^n\frac{\partial Q}{\partial \pi }& =\sum _j^n\frac{\partial }{\partial \pi }\left[m_j^{(i)}\left[\log Pi+\log p^{y_j}(1-p{)}^{1-y_j}\right]+\left(1-m_j^{\left(i\right)}\right)\cdot \left[\log (1-Pi)+\log q^{y_j}(1-q{)}^{1-y_j}\right]\right]\\ & =\sum _j^n\frac{\partial }{\partial \pi }\left[m_j^{\left(i\right)}\log Pi+(1-m_j^{\left(i\right)})\log (1-Pi)\right]\\ & =\sum _j^n\frac{m_j^{\left(i\right)}}{Pi}+\frac{m_j^{\left(i\right)}-1}{1-Pi}=0\\ & =\sum _j^n(1-p)m_j^{\left(i\right)}+Pi\left(m_j^{\left(i\right)}-1\right)=0\\ & \sum _j^n{m}_j^{\left(i\right)}=\sum _j^n\end{array}$$
即$$\sum _j^n{m}_j^{\left(i\right)}=n\pi$$事$$Pi=\frac{1}{n}\sum _j^n{m}_j^{\left(i\right)}$$
In the same steps, we respectively let ${\sum }_j^n\frac{\partial Q}{\partial p}=0$和${\sum }_j^n\frac{\partial Q}{\partial q}=0$ , you can find all the parameters of the next iteration$i=(Pi,p,q)$ can proceed to the next round of iterative optimization.

$$\begin{array}{rl}& \text{ make }\sum _j^n\frac{\partial Q}{\partial p}=0\\ & =\sum _j^n\frac{\partial }{\partial p}\left[m_j^{(i)}\left[\log Pi+\log p^{y_j}(1-p{)}^{1-y_j}\right]+\left(1-m_j^{\left(i\right)}\right)\cdot \left[\log (1-Pi)+\log q^{y_j}(1-q{)}^{1-y_j}\right]\right]\\ & =\sum _j^n\frac{\partial }{\partial p}\left[m_j^{\left(i\right)}\cdot \log p^{y_j}(1-p{)}^{(1-y_j)}\right]\\ & =\sum _j^n\frac{\partial }{\partial p}[y_j\cdot m_j^{\left(i\right)}\log p+m_j^{\left(i\right)}(1-y_j)\log (1-p)]\\ & =\sum _j^n\left[\frac{y_j\cdot m_j^{\left(i\right)}}{p}+\frac{m_j^{\left(i\right)}\cdot (1-y_j)\cdot (-1)}{1-p}\right]=0\\ & =\sum _j^n{y}_j\cdot m_j^{\left(i\right)}-m_j^{\left(i\right)}\cdot y_j\cdot p+p\cdot y_j\cdot m_j^{\left(i\right)}-p\cdot m_j^{\left(i\right)}=0\end{array}$$
即
$$\begin{gathered} \sum _j^n{y}_j\cdot m_j^{\left(i\right)}=\sum _j^{n}p\cdot m_j^{\left(i\right)} \\ p=\frac{{\sum }_j^n{y}_j\cdot m_j^{\left(i\right)}}{{\sum }_j^n{m}_j^{\left(i\right)}} \end{gathered}$$

接下来
$$\begin{array}{rl}& \text{ make }\sum _j^n\frac{\partial Q}{q}=0\\ & =\sum _j^n\frac{\partial }{\partial q}\left[m_j^{(i)}\left[\log Pi+\log p^{y_j}(1-p{)}^{1-y_j}\right]+\left(1-m_j^{\left(i\right)}\right)\cdot \left[\log (1-Pi)+\log q^{y_j}(1-q{)}^{1-y_j}\right]\right]\\ & =\sum _j^n\frac{\partial }{\partial q}\left[\left(1-m_j^{\left(i\right)}\right)\cdot \log q^{y_j}+\left(1-m_j^{\left(i\right)}\right)\cdot \left(1-y_j\right)\cdot \log (1-q)\right]\\ & =\sum _j^n\frac{\left(1-m_j^{\left(i\right)}\right)\cdot y_j}{q}+\frac{\left(1-m_j^{\left(i\right)}\right)\cdot \left(1-y_j\right)\cdot (-1)}{1-q}\\ & =\sum _j^n\left(y_j\cdot (1-q)-m_j^{\left(i\right)}\cdot y_j(1-q)+q\cdot \left(y_j-1\right)-q\cdot m_j^{\left(i\right)}\cdot \left(y_j-1\right)\right)\\ & =\sum _j^n\left(y_j-y_j\cdot q-m_j^{\left(i\right)}\cdot y_j+m_j^{\left(i\right)}\cdot y_j\cdot q+y_{j}q-q-m_j^{\left(i\right)}\cdot y_j\cdot q+m_j^{\left(i\right)}\cdot q\right)\\ & =\sum _j^n\left(y_j-m_j^{\left(i\right)}\cdot y_j-q+m_j^{\left(i\right)}\cdot q\right)\\ & \end{array}$$
即
$$\begin{array}{r}\sum _j^n{y}_j-m_j^{\left(i\right)}\cdot y_j=\sum _j^{n}q\left(1-m_j^{\left(i\right)}\right)\\ q=\frac{{\sum }_j^n\left(1-m_j^{\left(i\right)}\right)\cdot y_j}{{\sum }_j^n\left(1-m_j^{\left(i\right)}\right)}\end{array}$$

So far, we have obtained the parameters $i=(Pi,p,q)$分别为
$$\begin{gathered} Pi=\frac{1}{n}\sum _j^n{m}_j^{(i)} \\ p=\frac{{\sum }_j^n{y}_j\cdot m_j^{\left(i\right)}}{{\sum }_j^n{m}_j^{\left(i\right)}} \\ q=\frac{{\sum }_j^n\left(1-m_j^{\left(i\right)}\right)\cdot y_j}{{\sum }_j^n\left(1-m_j^{\left(i\right)}\right)} \end{gathered}$$

With the iteration relationship of the above parameters and samples
$$1,1,0,1,0,0,1,0,1,1We$$
set the initial parameters
$${Pi}^{(0)}=0.5,p^{\left(0\right)}=0.5,q^{\left(0\right)}=0.5$$
由$m_j^{\left(i\right)}$The expression gets $y_j=0$ or$y_j=1$，均有$m_j^{(1)}=0.5individual$
valuesπ
$$\begin{array}{rl}{Pi}^{(1)}& =\frac{1}{10}\left(m_1^{(0)}+m_2^{\left(0\right)}+m_3^{\left(0\right)}\cdots +m_{10}^{\left(0\right)}\right)\\ & =0.5\\ p^{\left(1\right)}& =\frac{y_1\cdot m_1^{\left(0\right)}+y_2\cdot m_2^{\left(0\right)}+y_3\cdot m_3^{\left(0\right)}+\cdots +y_{10}\cdot m_{10}^{\left(0\right)}}{\left(m_1^{\left(0\right)}+m_2^{\left(0\right)}+m_3^{\left(0\right)}\cdots +m_{10}^{\left(0\right)}\right)}\\ & =\frac{6\times 0.5}{10\times 0.5}\\ & =0.6\\ q^{\left(1\right)}& =\frac{\left(1-m_1^{\left(0\right)}\right)\cdot y_1+\left(1-m_2^{\left(0\right)}\right)\cdot y_2\cdots +\left(1-m_{10}^{\left(0\right)}\right)\cdot y_{10}}{\left(1-m_1^{\left(0\right)}\right)+\left(1-m_2^{\left(0\right)}\right)+\cdots +\left(1-m_{10}^{\left(0\right)}\right)}\\ & =\frac{6\times 0.5}{10-0.5\times 10}=0.6\end{array}$$
By continuing to iterate, we can get the final parameters $i=(Pi,p,A\; final\; value\; of\; q)$ isthe three-coin model.

Use Python code to conduct experimental simulations and check whether it is effective

The following is a piece of code that is used to generate experimental data and use the EM algorithm to estimate parameters. Use the estimated parameters to conduct multiple simulation experiments to test the model's prediction ability.

import random
import numpy as np
import time
def coin_experiment(pi, p, q, n):
    """
    模拟掷硬币实验
    pi: 硬币 A 正面出现的概率
    p: 硬币 B 正面出现的概率
    q: 硬币 C 正面出现的概率
    n: 实验次数
    """
    results = []
    results_A = []
    results_B = []
    results_C = []
    for i in range(n):
        # 先掷硬币 A
        if random.random() < pi:
            # 选硬币 B
            coin = 'B'
            p_head = p
        else:
            # 选硬币 C
            coin = 'C'
            p_head = q
        
        # 接着掷选出的硬币
        if random.random() < p_head:
            results.append(1)
        else:
            results.append(0)
        
        # 记录每个硬币的正反面
        if coin == 'B':
            if random.random() < p:
                results_B.append(1)
            else:
                results_B.append(0)
            results_A.append(1)
        else:
            if random.random() < q:
                results_C.append(1)
            else:
                results_C.append(0)
            results_A.append(0)
    
    # 计算 A、B、C 硬币的正面概率
    p_A = sum(results_A) / len((results_A))
    p_B = sum(results_B) / len(results_B)
    p_C = sum(results_C) / len(results_C)
    
    return results, p_A, p_B, p_C


pi = 0.2
p = 0.3
q = 0.8
n = 100000
s=time.time()
print(f'开始模拟，模拟参数为pi={
      
      pi},p={
      
      p},q={
      
      q}……')
Y,pi,p,q=coin_experiment(pi, p, q, n)
Y= np.array(Y)
print(f'模拟结束，共模拟{
      
      n}次，耗时{
      
      time.time()-s}……')
print(f"模拟结果，pi={
      
      pi},p={
      
      p},q={
      
      q},整体实验Y=1概率={
      
      sum(Y)/len(Y)}")

#EM参数反推，任意设定初始参数
pi_0 = 0.9
p_0 = 0.8
q_0 = 0.9
epsiodes=1000 #迭代次数
count=1
while count<=epsiodes:
    mu = pi_0 * p_0**Y * (1-p_0)**(1-Y) / (pi_0 * p_0**Y * (1-p_0)**(1-Y) + (1-pi_0) * q_0**Y * (1-q_0)**(1-Y))
    pi=(1/n)*sum(mu)
    p=sum(Y*mu)/sum(mu)
    q=sum((1-mu)*Y)/sum(1-mu)
    if count%100==0:
        print(f"第{
      
      count}次迭代，估算参数分别为：pi={
      
      pi},p={
      
      p},q={
      
      q}")
    pi_0 = pi
    p_0 = p
    q_0 = q
    count+=1

#用拿到的估计参数重新模拟下
s=time.time()
print(f'二次模拟检验，模拟参数为pi={
      
      pi},p={
      
      p},q={
      
      q}……')
Y,pi,p,q=coin_experiment(pi, p, q, n)
Y= np.array(Y)
print(f'模拟结束，共模拟{
      
      n}次，耗时{
      
      time.time()-s}……')
print(f"模拟结果，pi={
      
      pi},p={
      
      p},q={
      
      q},整体实验Y=1概率={
      
      sum(Y)/len(Y)}")

Observe the experiment and conduct 100,000 experiments according to the initial pi = 0.2, p = 0.3, q = 0.8 we set.

During the simulation process, pi, p, and q were very close to the values ​​we set. The experiment was indeed conducted according to these parameters. The probability of the overall experimental result y=1 was 0.696. Finally, the parameter pi = 0.91 was obtained after 1000 iterations of the EM algorithm. p = 0.68, q = 0.83. Although they are very different from the actual parameters of our model, we conducted a new 100,000 experiments with these parameters. The final overall experimental result is that the probability of y=1 is 0.6983, which is very similar to the results of our actual model. It is close, a more accurate simulation is achieved, and the model is effective .

References

[1]. Detailed explanation of the derivation process of Q function of EM algorithm