"Machine Learning Formula Derivation and Code Implementation" chapter22-EM algorithm

"Machine Learning Formula Derivation and Code Implementation" study notes, record your own learning process, please buy the author's book for detailed content.

EM algorithm

As an iterative algorithm, the EM algorithm ( expectation maximization, 期望极大值算法) is used for the maximum likelihood estimation of the parameters of the probability model including hidden variables .

EM algorithm includes two steps: E step, seeking expectation ( expectation); M step, seeking maximum ( maximization).

1 Maximum Likelihood Estimation

Maximum likelihood estimation ( maximum likelihood estimation, MLE) is a classic parameter estimation method in the field of statistics. For a random sample that satisfies a certain probability distribution, but the statistical parameters in it are unknown, the maximum likelihood estimation allows me to estimate the value of the parameter through the results of several experiments.

To illustrate with a classic example, for example, we want to know the height distribution of students in a certain college. We assume that the height distribution of the students in this school obeys a normal distribution $N(\mu ,p^2)$, where the distribution parameter$\mu$和$p^2$ unknown. There are tens of thousands of students in the school, and it is definitely unrealistic to measure them one by one, so we decided to use the statistical sampling method to randomly select 100 students to measure their height.

To estimate the height of all students in the school through the height of 100 people, the following questions need to be clarified. The first question is what is the probability of drawing these 100 people . Because everyone is selected independently, the probability of drawing 100 people can be expressed as the product of individual probabilities:
$$L\left(i\right)=L(x_1,x_2,\cdots ,x_n;i)=\prod _{i=1}^{n}p(x_i\mid \theta )$$
The above formula is the likelihood function, for the convenience of calculation, we will take the logarithm of the likelihood function:
$$H\left(i\right)={\ln }_{}L\left(i\right)={\ln }_{}\prod _{i=1}^{n}p(x_i\mid i)=\sum _{i=1}^n{\ln }_{}p(x_i|\theta )$$
The second question is why these 100 people were just selected. According to the theory of maximum likelihood estimation, among so many students in the school, we just selected these 100 students instead of the other 100 students, precisely because the probability of these 100 students appearing is extremely high, that is, the corresponding likelihood function is extremely large:
$$\hat{i}=argmaxxL\left(\theta \right)$$
How to solve the last problem, directly for$L\left(\theta \right)$ is differentiated and made to be 0.

Therefore, the maximum likelihood estimation method can be regarded as the inverse deduction of the conditions from the sampling results , that is, a certain parameter is known to make the probability of these samples appear extremely high, and we directly use this parameter as the true value of the parameter estimation.

2 EM algorithm

Assuming that the height of all students in the school pays a distribution is too general, in fact, the distribution of men and women is different, assuming that the height of boys obeys the distribution $N(m_1^{},p_1^2)$ , the height distribution of girls is$N(m_2^{},p_2^2)$ . Now to estimate the height of the students in this school, we cannot simply use a distribution assumption.

Assume that 50 boys and 50 girls are sampled separately, and they are estimated separately. Suppose we don't know whether the samples we sampled come from boys or girls.

The student's height is an observed variable ( observable variable), and the gender of the sample is a hidden variable ( hidden variable).

Now we need to estimate two questions: one is whether the sample is a boy or a girl, but what are the normal distribution parameters of the heights corresponding to boys and girls. In this case, the maximum likelihood estimation is not applicable. To estimate the height distribution of male and female students, it is necessary to first estimate whether the student is male or female. Conversely, to estimate whether the student is male or female, it must be judged from the height. However, the two depend on each other, and cannot be calculated directly by maximum likelihood estimation.

For this parameter estimation problem involving hidden variables, the EM( expectation maximization) algorithm is generally used to solve it, that is, the expectation maximization algorithm. For the above-mentioned height estimation problem, the solution idea of ​​the EM algorithm is: since the two problems depend on each other, this must be a dynamic solution process. It is better to directly give the initial value of the distribution of male and female heights, estimate the probability of which sample is male/female according to the initial value ( E step ), and then use the maximum likelihood to estimate the height distribution parameters of male and female students ( M step ), and then dynamically Iterative adjustments are made until the termination condition is met.

The application scenario of the EM algorithm is to solve the parameter estimation problem of a probability model containing hidden variables. Given the observed variable data Y , the hidden variable data Z , the joint probability distribution $P(Y,Z|\theta )$ , andthe conditional distribution on hidden variables$P(Z|Y,\theta )$ , use EM algorithm to model parametersθ \thetaThe process of estimating$\theta$
is as follows: (1) Initialize model parameters$i^{\left(0\right)}$ , start iteration.
(2) Step E: remember$i^{\left(i\right)}$ for$i$ iteration parameterθ \thetaEstimated value of$\theta$$i+1$ Define the E function, solve the function Q:
$$Q(\theta ,i^{(i)})=E_Z\left[{\log }_{}P(Y,Z\mid i)|Y,i^{(i)}\right]=\sum _Z{\log }_{}P(Y,Z\mid i)P(Z|Y,i^{\left(i\right)})$$
of which$P(Z|Y,i^{\left(i\right)})$is the given observation data$Y$ and the current parameter estimate$i^{In\; the\; case\; of\; (i)}$ , hidden variable dataZZThe conditional probability distribution of$Z.$The key to the E step is this Q function, which is defined as the logarithmic likelihood function${\log }_{}P(Y,Z|\theta )$ about the given observation data$Y$ and the current parameter$i^{In\; the\; case\; of\; (i)}$ , the unobserved dataZZThe conditional probability distribution of$Z.$(3) M step: Find the parameter θ \theta
that maximizes the Q function$\theta$ , determine the$i+Parameter\; estimates\; for\; 1$ iteration$i^{(i+1)}$：
$$i^{(i+1)}=\underset{i}{\mathrm{argmax}}Q(\theta ,i^{\left(i\right)})$$
(4) Repeat step E and step M until convergence.

It can be seen from the EM algorithm process that the key is to determine the Q function in the E step. The E step estimates the hidden state variable distribution under the condition of fixing the model parameters, while the M step estimates the model parameters by fixing the hidden variables. The two are carried out interactively until the convergence condition of the algorithm.

EM algorithm dynamic iterative process:

3 Three-coin model

4 Realize the three-coin model based on numpy

import numpy as np

## EM算法过程定义
def em(data, thetas, max_iter=50, eps=1e-3): # data观测数据，thetas初始化的估计参数值，eps收敛阈值

    ll_old = 0 # 初始化似然函数值
    for i in range(max_iter):
        # E步：求隐变量分布
        log_like = np.array([np.sum(data*np.log(theta), axis=1) for theta in thetas]) # 对数似然 2*5
        like = np.exp(log_like) # 似然 2*5
        ws = like/like.sum(0) # 隐变量分布 2*5
        ll_new = np.sum([w*l for w, l in zip(ws, log_like)]) # 期望

        # M步：更新参数值
        vs = np.array([w[:, None] * data for w in ws]) # 概率加权 2*5*2
        thetas = np.array([v.sum(0)/v.sum() for v in vs]) # 2*2

        # 打印结果
        print(f'Iteration:{
      
      i+1}')
        print(f'theta_B = {
      
      thetas[0,0]:.2}, theta_C = {
      
      thetas[1,0]:.2}, {
      
      ll_new}')

        # 满足条件退出迭代
        if np.abs(ll_new - ll_old) < eps:
            break
        ll_old = ll_new
    
    return thetas

The EM algorithm solves the three-coin problem:

# 观测数据，5次独立实验，每次试验10次抛掷的正反面次数
observed_data = np.array([(5, 5), (9, 1), (8, 2), (4, 6), (7, 3)]) # 比如第一次试验为5次正面5次反面
# 初始化参数值，硬币B出现正面的概率0.6，硬币C出现正面的概率为0.5
thetas = np.array([[0.6, 0.4], [0.5, 0.5]])
# EM算法寻优
thetas = em(observed_data, thetas, max_iter=30)
thetas

Iteration:1
theta_B = 0.71, theta_C = 0.58, -32.68721052517165
Iteration:2
theta_B = 0.75, theta_C = 0.57, -31.258877917413145
Iteration:3
theta_B = 0.77, theta_C = 0.55, -30.760072598843628
Iteration:4
theta_B = 0.78, theta_C = 0.53, -30.33053606687176
Iteration:5
theta_B = 0.79, theta_C = 0.53, -30.071062062760774
Iteration:6
theta_B = 0.79, theta_C = 0.52, -29.95042921516964
Iteration:7
theta_B = 0.8, theta_C = 0.52, -29.90079955867412
Iteration:8
theta_B = 0.8, theta_C = 0.52, -29.881202814860167
Iteration:9
theta_B = 0.8, theta_C = 0.52, -29.873553692091832
Iteration:10
theta_B = 0.8, theta_C = 0.52, -29.870576075992844
Iteration:11
theta_B = 0.8, theta_C = 0.52, -29.86941691676721
Iteration:12
theta_B = 0.8, theta_C = 0.52, -29.868965223428773

array([[0.7967829 , 0.2032171 ],
       [0.51959543, 0.48040457]])

The algorithm converges at the seventh iteration, and finally the probabilities of coin B and coin C appearing heads are 0.80 and 0.52, respectively.

Notebook_Github address