Part11. Arithmetic operations on images
The essence of an image is a matrix, so some common arithmetic operations can be performed on it, such as addition, subtraction, multiplication, division, square root, logarithm, absolute value, etc. In addition, logical operations and geometric transformations can also be performed on images.
Let's start with simple image addition, subtraction, and logical operations. There will be special content to introduce the geometric transformation of images in the future.
11.1 Image addition
Image addition is to add two images of the same size and type pixel by pixel, and finally obtain a new image.
Image addition, subtraction, multiplication, and division operations are performed on two images of the same size and type .
1.1.1 Example of addition
Formula for adding images:
You can also use: dst += src1, where += is a C++ overloadable operator.
Give a simple example:
Mat a = imread(".../cat.jpg");// 加载了一张猫的图片
imshow("a", a);
Mat b = Mat(Size(a.cols,a.rows),a.type(), Scalar(0,0,255));// 生成跟a大小类型一样，红色的图像
Mat c;
cv::add(a,b,c);// 将 a、b 相加，结果为c
imshow("c", c);
In the above code, the Mat object c is the product of the addition of Mat objects a and b. If b is changed to white, it is Scalar(255,255,255). So what will c become? The answer is still white. Because addition is the sum of pixels, if two pixels exceed 255, it will still become 255.
1.1.2 Implement the function of add() function
In order to explain the above problem, we try to implement the function of an add function ourselves.
Mat a = imread(".../cat.jpg"); // 加载 cat 的图像
imshow("a", a);
Mat b = Mat(Size(a.cols,a.rows),a.type(), Scalar(0,0,255));
int h = a.rows; // 图像 a 的高
int w = a.cols; // 图像 a 的宽
Mat c = Mat::zeros(a.size(), a.type());
for (int row = 0; row < h; row++)
{
for (int col = 0; col < w; col++)
{
Vec3b p1 = a.at<Vec3b>(row, col);
Vec3b p2 = b.at<Vec3b>(row, col);
c.at<Vec3b>(row, col)[0] = saturate_cast<uchar>(p1[0] + p2[0]);
c.at<Vec3b>(row, col)[1] = saturate_cast<uchar>(p1[1] + p2[1]);
c.at<Vec3b>(row, col)[2] = saturate_cast<uchar>(p1[2] + p2[2]);
}
}
imshow("c", c);Each pixel of images a and b is traversed through a two-layer for loop, and the results are added and assigned to the corresponding pixel of image c. When adding, the saturate_cast() function is used.
saturate_cast() is a template function whose function is to prevent overflow . It supports uchar, short, int, float, double and other types.
For uchar type, if the pixel value exceeds 255, its value becomes 255 after using the saturate_cast() function. This also just explains that if b is white, then the final c object will also be white.
1.1.3 Image overlay using copyTo() function
In the previous article, we introduced the copyTo() function, which can copy a Mat object to another Mat object.
Now let’s review its use
Mat a = imread(".../cat.jpg"); // 加载 cat 的图像
Mat b = imread(".../leaf.png"); // 加载一张小尺寸的树叶的图像
Mat roi = a(Rect(0,0,b.cols,b.rows));
b.copyTo(roi);
imshow("result", a);In the above code, the roi object intercepts an area from the a object, and the area is the same size as the b object. Since the operation of extracting roi is a shallow copy , after copying the b object to the roi object, the a object itself will be changed.
The following is the result of execution:
Therefore, the overlay of images can be achieved with the help of the copyTo() function.
21.2 Linear blending of images (linear blending)
Linear blending formula for images: dst = src1 * alpha + src2 * beta + gamma
Among them, alpha and beta represent the weights of image 1 and image 2 respectively, and gamma is the brightness adjustment amount . When alpha = beta = 1 and gamma = 0, it represents the addition of two images.
The two images for linear blending must also be of the same size and type.
Mat a = imread(".../cat.jpg"); // 加载 cat 的图像
Mat b = imread(".../chinese_flag.png"); // 加载五星红旗的图像
resize(a, a,Size(b.cols,b.rows));// 缩放a的大小，跟b保持一致
Mat dst;
addWeighted(a, 0.5, b, 0.5,0, dst);
imshow("dst", dst);Since the sizes of images a and b are different, the resize() function needs to be used to scale the size of image a according to the size of image b before linear mixing.
The above code completes a linear mixture of cats and five-star red flags. If you still want to try to make a National Day version of the gradient avatar, you need to get closer to the red flag, and the greater the weight of the red flag.
We can write the code like this:
Mat a = imread(".../cat.jpg"); // 加载 cat 的图像
Mat flag = imread(".../chinese_flag.png");
int flag_width = flag.cols;
int flag_height = flag.rows;
Mat dst;
resize(a, dst, Size(flag_width, flag_height));
int radius = 0;
if (flag_width > flag_height) {
radius = flag_width;
} else {
radius = flag_height;
}
for (int i=0; i < dst.rows; i++) {
for (int j=0; j < dst.cols; j++) {
int distance = std::sqrt(i*i+j*j);
double alpha;
if (distance > radius) {
alpha = 1;
} else {
alpha = (double) distance / radius;
}
double beta = 1 - alpha;
Vec3b v1 = dst.at<Vec3b>(i, j);
dst.at<Vec3b>(i, j)[0]= alpha * v1[0] + beta * flag.at<Vec3b>(i, j)[0];
dst.at<Vec3b>(i, j)[1]= alpha * v1[1] + beta * flag.at<Vec3b>(i, j)[1];
dst.at<Vec3b>(i, j)[2]= alpha * v1[2] + beta * flag.at<Vec3b>(i, j)[2];
}
}
imshow("dst", dst);
31.3 Image subtraction
Image subtraction is the subtraction of two images pixel by pixel. Image subtraction can detect the difference between the two images. This difference can be used to perform various detections, so image subtraction has practical uses in many fields.
The formula for image subtraction:
You can also use: dst -= src1, where -= is a C++ overloadable operator.
Give a simple example:
Mat a = imread(".../cat.jpg"); // 加载 cat 的图像
int width = a.cols;
int height = a.rows;
Mat b = Mat(Size(width,height), a.type(),Scalar(0,0,0));
circle(b, Point(width/2, height/2), 600, Scalar(255,255,255), -1);
Mat dst;
subtract(a,b,dst);
imshow("dst", dst);
The result of the above execution is the result of subtracting image b from image a, and the cat in the middle is "cut out". What if I only want the cat in the middle and not the background? This article will use the bitwise_and operation to obtain it later.
For another example, perform Gaussian blur on the loaded image , and then subtract the Gaussian blur from the original image to get the difference between the two images.
Mat a = imread(".../cat.jpg"); // 加载 cat 的图像
imshow("a",a);
Mat b;
GaussianBlur(a, b,Size(15,15),0,0);
imshow("b",b);
Mat dst;
subtract(a,b,dst);
imshow("dst",dst);
After the introduction of image subtraction, the usage of image multiplication (multiply), division (divide), and absolute difference (absdiff) are very similar, and they are often used in actual work. In particular, the absdiff() function is represented by the formula:
It can be used to obtain difference maps , which are often used in video analysis.Part22. Logical operations on images
42.1 Basic knowledge of masks
Before introducing the logical operations of images, let's review the knowledge of masks, because the mask parameter is used in many functions of OpenCV.
Both arithmetic operations and logical operations on images support masks.
The mask is a single-channel matrix smaller than or equal to the source image , and the values in the mask are divided into 0 and non-0.
Image masking uses selected images, graphics or objects to block the processed image (all or part of it) to control the image processing area or process.
The function of mask:
Extract ROI
shielding effect
Extract result features
Create specially shaped images
There are many ways to generate masks.
We can create one ourselves by slightly changing the first example of image subtraction, image b. Because mask is a single-channel matrix.
Mat mask = Mat(Size(width,height), CV_8UC1,Scalar(0,0,0));
circle(mask, Point(width/2, height/2), 600, Scalar(255,255,255), -1);We can also extract the mask through image binarization threshold segmentation , for example:
Mat src = imread(".../leaf.png"); // 加载一张小尺寸的树叶的图像
imshow("src",src);
Mat gray;
cvtColor(src,gray,COLOR_BGR2GRAY);
Mat mask;
threshold(gray, mask, 0, 255, THRESH_BINARY_INV|THRESH_OTSU);
imshow("mask",mask);
The relevant content of image binarization will be specifically introduced in subsequent articles. In short, there are many ways to create masks.
52.2 Logical operations
Two images can be subjected to logical operations such as AND, OR, XOR, etc. Here is the truth table for logical operations:
| a | b | a AND b | a OR b | a XOR b | NOT a |
|---|---|---|---|---|---|
| 0 | 0 | 0 | 0 | 0 | 1 |
| 0 | 1 | 0 | 1 | 1 | 1 |
| 1 | 0 | 0 | 1 | 1 | 0 |
| 1 | 1 | 1 | 1 | 0 | 0 |
in,
The principle of the AND operation: If the two values a and b have 0, the result of the AND is 0; if a and b are both 1, the result of the AND is 1.
The principle of OR operation: If the two values of a and b have 1, the result of OR is 1; if a and b are both 0, the result of AND or is 0.
The principle of XOR operation: If the two values a and b are not the same, the XOR result is 1; if the two values a and b are the same, the XOR result is 0.
The principle of NOT operation: If the value of a is 1, the result of NOT operation is 0; if the value of a is 0, the result of NOT operation is 1.
Logical operations on images also require two images of the same size and type to perform operations.
Mat a = imread(".../cat.jpg"); // 加载 cat 的图像
Mat b = Mat(Size(a.cols,a.rows),a.type(), Scalar(0,0,255));// 生成跟a大小类型一样，红色的图像
Mat dst1,dst2,dst3,dst4;
bitwise_and(a,b,dst1);
bitwise_or(a,b,dst2);
bitwise_xor(a,b,dst3);
bitwise_not(a,dst4);
imshow("bitwise_and", dst1);
imshow("bitwise_or", dst2);
imshow("bitwise_xor", dst3);
imshow("bitwise_not", dst4);
The functions corresponding to the logical AND, OR, XOR, and NOT operations in OpenCV are bitwise_and, bitwise_or, bitwise_xor, and bitwise_not respectively. The above figure also shows the execution results of these functions respectively.
Now let's answer the previous question, how to "pick out" only the cat in the middle? The answer is just use the bitwise_and function.
Mat a = imread(".../cat.jpg"); // 加载 cat 的图像
int width = a.cols;
int height = a.rows;
Mat b = Mat(Size(width,height), a.type(),Scalar(0,0,0));
circle(b, Point(width/2, height/2), 600, Scalar(255,255,255), -1);
Mat dst;
bitwise_and(a,b,dst);
imshow("dst", dst);
62.3 Use mask for image fusion
Make a slight change to the code just now, change the type of image b to CV_8UC1, and rename it to mask. The usage of bitwise_and function has also been slightly adjusted. When mask participates in bitwise_and operation, the execution result is the same as before.
Mat a = imread(".../cat.jpg"); // 加载 cat 的图像
int width = a.cols;
int height = a.rows;
Mat mask = Mat(Size(width,height), CV_8UC1,Scalar(0,0,0));
circle(mask, Point(width/2, height/2), 600, Scalar(255,255,255), -1);
Mat dst;
bitwise_and(a,a, dst,mask);
imshow("dst", dst);Because, when the bitwise_and function uses the mask parameter, the operation will only be performed on pixels with non-empty mask values. So it can be used to remove the background and extract ROI.
Use mask to perform a "logical AND" operation, that is, the white area of the mask image is to retain the pixels of the image that need to be processed , and the black area is to eliminate the pixels of the image that need to be processed . The principles of other logical operations are similar but the effects are different.
The image generated by the image superposition implemented previously using the copyTo() function does not produce ideal results because the leaves are not transparent.
Next, try to perfectly blend the two images.
Mat a = imread(".../cat.jpg"); // 加载 cat 的图像
Mat b = imread(".../leaf.png"); // 加载一张小尺寸的树叶的图像
Mat b2gray;
cvtColor(b,b2gray,COLOR_BGR2GRAY); // 对 b 转换成灰度图像
imshow("b2gray", b2gray);
Mat mask,mask_inv;
threshold(b2gray, mask, 0, 255, THRESH_BINARY_INV|THRESH_OTSU);// 二值分割获取 mask
imshow("mask", mask);
bitwise_not(mask,mask_inv);
imshow("mask_inv", mask_inv);
Mat roi = a(Rect(0,0,b.cols,b.rows));
Mat fg,bg;
bitwise_and(roi,roi,bg, mask_inv);
imshow("bg", bg); // 提取 roi 的背景
bitwise_and(b,b,fg,mask);
imshow("fg", fg); // 提取 b 的前景
Mat dst;
add(bg,fg,dst);
dst.copyTo(roi);
imshow("result", a);First load two images, respectively a and b objects.
Convert the b object into a grayscale image, then obtain the mask through binary segmentation, and perform a NOT operation on the mask to obtain mask_inv.
Perform the interception of roi on object a, and the size of roi will be the same as that of object b.
Then use AND operation to extract the background of roi and the foreground of object b respectively. Add the two and copy the result to the roi object. Finally, we can see the result of the perfect fusion of the two images.
The following pictures show the objects generated at each stage of the code, as well as the final result.
Part33. Summary
This article is divided into two parts. The first part introduces the arithmetic operations of images, mainly introducing image addition, subtraction, their implementation principles and usage scenarios, and also introduces the linear mixing of images.
The second part introduces the logical operations of images, reviews the purpose of mask, and how to use mask in the bitwise_and function.