C ++-OpenCV pit

　　I stepped on the opencv pit for 1 day and couldn't help vomiting. . .

　　Normal two matrix operations cannot be applied to a series of 8U images. This is the biggest pit. For example, if you want to add two image matrices:

Mat im1 = imread("../opencv/samples/data/pic5.png", CV_8UC1);
Mat im2 = imread("../opencv/samples/data/pic6.png", CV_8UC1);
Mat overlap = im1 + im2;

　　If you write this, the resulting image is absolutely wrong! ! ! You have to do some special conversions.

　　Well, after running it first, you will see the result:

　　Here im1 and im2 are like this (the sample image that comes with opencv):

　　(IM1)

　　(im2)

　　At first glance, it seems to be right, but in fact it is correct (running in python opencv):

　　The reason for this in C ++ is that the value range of the 8U image is 0 ~ 255, and when Mat is added, those greater than 255 will overflow. . Then the value of the pixels that overflowed due to the addition will be 255. At first I wrote this method to correct:

Mat matr_add(Mat im1, Mat im2)
{
    CV_Assert(im1.size() == im2.size());
    Mat uxx = Mat::zeros(im1.size(), CV_8UC1);
    for (int i = 0; i < uxx.rows; i++)
    {
        for (int j = 0; j < uxx.cols; j++)
        {
            uxx.at<uchar>(i, j) = (im1.at<uchar>(i, j) + im2.at<uchar>(i, j)) % 256;
        }
    }
    return uxx;
}

　　operation result:

　　Correct.

　　This is more obvious when performing matrix multiplication (C ++ opencv):

　　And correct (running on python opencv):

　　Also use the method of modulo correction:

Mat multiply_mod(Mat im1, Mat im2)
{
    CV_Assert(im1.size() == im2.size());
    Mat uxx = Mat::zeros(im1.size(), CV_8UC1);
    for (int i = 0; i < uxx.rows; i++)
    {
        for (int j = 0; j < uxx.cols; j++)
        {
            uxx.at<uchar>(i, j) = (im1.at<uchar>(i, j) * im2.at<uchar>(i, j)) % 256;
        }
    }
    return uxx;
}

　　operation result:

　　Fixed it.

　　In the same way, subtraction also needs to be repaired:

Mat matr_sub(Mat im1, Mat im2)
{
    CV_Assert(im1.size() == im2.size());
    Mat uxx = Mat::zeros(im1.size(), CV_8UC1);
    for (int i = 0; i < uxx.rows; i++)
    {
        for (int j = 0; j < uxx.cols; j++)
        {
            uxx.ptr<uchar>(i)[j] = (im1.ptr<uchar>(i)[j] - im2.ptr<uchar>(i)[j]) < 0 ? (im1.ptr<uchar>(i)[j] - im2.ptr<uchar>(i)[j]) % 256 : (im1.ptr<uchar>(i)[j] - im2.ptr<uchar>(i)[j]);
        }
    }
    return uxx;
}

　　Very beautiful, it seems to solve all problems.

　　But in fact, the problem is not so simple, there is still a question how to fix the division, because in some cases, after doing some operations on the image, you will get a 0 (the reason is also because 8UC1 type Mat is used to store pixels, so The value of 0.xxx is directly treated as 0). At this time, I ca n’t fix it, but I want to obtain or set the correct floating-point pixel value. This problem directly leads to the image similarity detection I wrote. The algorithm doesn't work properly ...

　　So, rethink calmly.

　　The first thing to be clear is that directly using CV_32FC1 to read the image is not acceptable. Unexpectedly, many Nan or inf values will be obtained, so the image cannot be displayed normally.

　　And we must not use the following codes to access pixel elements (the following assumes the use of C1 single-channel images):

　　Case 1: Assume the im type is CV_8UC1

im.at<int>(row, col);
im.at<float>(row, col);
//...

　　Otherwise, you will find that your image is 600 x 600, but you read a small part, and even report the assertion interrupt error of opencv_assert as soon as you enter the loop. See: https://github.com/kinchungwong/kinchungwong.github.io/blob/master/opencv_issues_11370/explain_opencv_non_issue_11370.md

　　For the 8U series, you can only use this method:

im.at<uchar>(row, col)
im.ptr<T>(row)[col];

　　In other cases, for other types of Mat, it is better to get or set pixel element information in this way:

im.ptr<T>(row)[col];

　　Never use at <T> () to access, because it calculates memory address by type, so it will definitely exceed the range. The program will be interrupted immediately.

　　Back to the topic.

　　In fact, we want to ensure the correct and convenient matrix operation of the image. In fact, there is only one step to do. First, we use CV_8UC1 (using a single-channel image as an example) to read the image, and then convert its matrix type to CV_32FC1. Now! It ’s that simple but it ’s been me for a long time:

Mat im1 = imread("../opencv/samples/data/lena.jpg", CV_8UC1);
Mat im1f;

im1.convertTo(im1f, CV_32FC1);
// ...

　　Therefore, the SSIM algorithm I wrote can be used. In the next article, I will fully introduce the SSIM algorithm ~

Guess you like