Digital Image Processing (12) Maximum Entropy Algorithm

---

foreword

In image analysis, it is usually necessary to extract the target of interest from the image. This process of separating and extracting a specific area from other parts of the image is image segmentation . Therefore, image segmentation processing is actually to distinguish the "foreground object" and "background" in the image, so it is usually called binary processing of the image . We have introduced the threshold method based on image gray distribution and the Otsu binarization algorithm before . Today we introduce another binarization algorithm: the maximum entropy method.

1. What is entropy?

  Entropy is a measure of uncertainty in information theory and a measure of the amount of information contained in data. When the entropy takes the maximum value, it indicates that the amount of information obtained is the largest.
Information volume: The amount of information varies from large to small. For example, the sun rises from the east, which is a definite event without any amount of information; for example, if someone buys a lottery ticket and has a 99% probability of winning, then this event has a lot of information, because the probability is very high. Little things become sure.

1. How to calculate the amount of information?

  The information content of an event is the negative logarithm of the probability of the event occurring.
For example, the symbol $The\; probability\; of\; x$ occurring is$p\left(x\right)$ , then the symbolxxSelf-information of$x$$I$为$$I=-logp(x)$$

2. How is entropy calculated?

  Suppose the symbol set has $n$ symbols, the probability of each symbol appearing is$p(x_i)$ , then the information entropy of the symbol set is$$H=-\sum _{i=1}^{n}p(x_i)logp(x_i)$$
can get a conclusion:when$p(x_1)=p\left(x_2\right)=\cdots =p\left(x_n\right)$ entropy takes the maximum value.

Second, the maximum entropy method

1. Design thinking

The design idea of ​​the maximum entropy method is: select an appropriate threshold to divide the image into two categories, and when the sum of the average entropy of the two categories is the largest, the maximum amount of information can be obtained from the image to determine the optimal threshold.

2. Algorithm steps

1. Find the distribution probability of all pixels in the image $p_0,p_1,\cdots ,p_{255}$(The grayscale distribution range of the image is [0,255]) $$p_i=\frac{N_i}{N_{image}}$$Among them, $N_i$is the gray value $The\; number\; of\; pixels\; of\; i$ ;$N_{image}$is the total number of pixels of the image;
2. Given an initial threshold $Th=T{h}_0$，$Th\in [0,255]$ ; Divide the image into$C_1$and $C_2$two types;
3. Calculate the average relative entropy $$E_1=-\sum _{i=0}^{Th}(p_i/p_{Th})\cdot ln(p_i/p_{Th})$$$$E_2=-\sum _{i=Th+1}^{255}(pi/(1-p_{Th}))\cdot ln(p_i/(1-p_{Th}))$$其中$p_{Th}={\sum }_{i=0}^{Th}{p}_i$；
4. This $E_1+E_2$When the sum is the maximum value, $Th$ is the optimal threshold$T{h}^{\ast }$ , which satisfies the maximum amount of information in the image at this time.

3. C++ code

/* 最大熵算法 */
int main()
{
    
    
    cv::Mat image = cv::imread("Lena.bmp");
    cv::Mat gray_image = cv::Mat::zeros(image.size(), CV_8UC1);
    cv::cvtColor(image, gray_image, cv::COLOR_BGR2GRAY);

    int height = gray_image.rows;
    int width = gray_image.cols;

    // 计算像素分布概率
    float p[256] = {
    
     0 };
    for (int row = 0; row < height; row++)
    {
    
    
        for (int col = 0; col < width; col++)
        {
    
    
            p[gray_image.at<uchar>(row, col)] = p[gray_image.at<uchar>(row, col)] + 1;
        }
    }
    for (int i = 0; i < 256; i++)
    {
    
    
        p[i] = p[i] / (height*width);
    }
    //保存E_1+E_2
    float E[256] = {
    
     0.0 };

    //遍历所有的像素值
    int index = 0; //最大值的索引
    for (int th = 0; th < 256; th++)
    {
    
    
        // 计算p_Th
        float p_Th = 0.0;
        for (int i = 0; i < th+1; i++)
        {
    
    
            p_Th += p[i];
        }

        // 计算E_1
        float E_1 = 0.0, E_2 = 0.0;

        for (int i = 0; i < th + 1; i++)
        {
    
    
            if (fabs(p_Th) < 1e-6)
            {
    
    
                E_1 = 0;
            }
            else
            {
    
    
                E_1 += -(p[i] / p_Th) * log(p[i] / p_Th + 1e-6);
            }
            
        }
        for (int j = th + 1; j < 256; j++)
        {
    
    
            if (fabs(1 - p_Th) < 1e-6)
            {
    
    
                E_2 = 0;
            }
            else
            {
    
    
                E_2 += -(p[j] / (1 - p_Th))*log(p[j] / (1 - p_Th)+1e-6);
            }
        }
        if ((E_1 + E_2) > E[index])
        {
    
    
            index = th;
        }
        E[th] = E_1+E_2;
    }

    // 进行二值化
    cv::Mat output_image = cv::Mat::zeros(height, width, CV_8UC1);
    for (int row = 0; row < height; row++)
    {
    
    
        for (int col = 0; col < width; col++)
        {
    
    
            if (gray_image.at<uchar>(row, col) > index)
            {
    
    
                output_image.at<uchar>(row, col) = 255;
            }
            else
            {
    
    
                output_image.at<uchar>(row, col) = 0;
            }
        }
    }
    cv::imshow("input-image", gray_image);
    cv::imshow("output-image", output_image);
    cv::waitKey(0);
    return 0;
}

A few notes about the code:

1. Because the divisor may be 0, it is necessary to judge p_Th as 0. You can refer to C++ float and double to judge whether it is equal to 0 , and the same is true for 1-p_Th.
2. Because the log() function may take 0 in the brackets, resulting in negative infinity, so add a small integer 1e-6 to avoid negative infinity.

4. Experimental results

References

1. Fundamentals of digital image processing. Zhu Hong.