This chapter mainly introduces a relatively basic operation in image processing: Canny edge discovery, contour discovery and contour drawing. The concept is not difficult. It mainly combines the API related operations of OpenCV 4.5+ to prepare knowledge for the next "watershed image segmentation based on distance transformation".
Canny edge detection
Before talking about contours, it is necessary to take some time to learn a famous algorithm for edge detection and extraction-Canny edge extraction algorithm. The significant difference in the edge detection effect of this algorithm compared to other edge detection algorithms is that the edges detected by Canny are relatively thin and clear. Compared with Sobel and Laplace that we learned before, this algorithm is an application method that truly "extracts" edges; while Sobel and Laplace only extract them from the collection of image pixels.
The edge detection and extraction of the Canny algorithm mainly includes the following steps:
1. Grayscale cvColor and Gaussian filtering GaussianBlur
Turn the image into a grayscale image and reduce the channels. The function of Gaussian filter is to smooth the image and reduce noise, so that it will not be mistaken for edges when detected by the Canny algorithm, so use this Gaussian filter at the beginning to reduce the more prominent places. To put it simply, it is to filter out inappropriate places on the image.
2. Calculate the gradient and gradient direction of the image Sobel/Scharr
The edge of the image is where the gray value changes sharply. For example, in a grayscale image, there are only changes in light and dark. When the intensity change in a certain place is relatively drastic, it will form an edge. Where there are large changes in light and dark, the gradient changes will also be large.
3. Non-maximum suppression
What is this step going to do? After the above operation, we only enhanced the image, rather than finding the real edge. Moreover, the edge discovered after steps 1-2 is a range signal, but the edge can only be one line or a cluster, so I need to suppress and eliminate non-edge individuals? How to do it is to remove it if it is not the given maximum value in the gradient direction of the edge. (As shown in the figure below, the left side is Sobel's gradient direction, and the right side is the commonly used suppression range selection)
But how to set this maximum threshold? If it is too high, many of the positions that were originally lines will be unset. If it is too low, the edges will be broken. We hope that the effect is to find clear and continuous edge lines.
4. Double threshold filtering edge connection
After non-maximum suppression, the image detected edges still have many grays and are not clear. So next, set up a double threshold and specify the upper and lower thresholds. The so-called double threshold means that there are two thresholds, namely the low threshold and the high threshold. If the pixel gray value is greater than the maximum threshold, it is directly updated to 255. If the gray value of a pixel is less than the minimum threshold, its gray value is updated to 0. If the gray value of a pixel is between the maximum threshold and the minimum threshold, it depends on whether there is a value greater than the maximum threshold in its 8 neighborhoods. If so, it will be classified as 255, or the average of the 8 areas can be taken.
The Canny edge detection algorithm basically goes through the above steps. OpenCV has a corresponding optimized Canny method. Let’s take a look at how to use it. It will be used when learning to find contours.
CV_EXPORTS_W void Canny(
InputArray image, // 8-bit输入图像
OutputArray edges, // 输出的边缘图像，一般都是二值图像，背景是黑色
double threshold1, // 低阈值，常取高阈值的1/2或者1/3
double threshold2, // 高阈值
int apertureSize = 3, // Sobel算子的size，取值3代表是3x3
bool L2gradient = false // 选择true用L2=sqrt{(dI/dx)^2 + (dI/dy)^2}求梯度方向，
// 默认false用L1=|dI/dx|+|dI/dy|计算
);contour
- Contour discovery is a method of finding object contours based on image edge extraction. Therefore, the selected threshold for edge extraction will affect the final contour discovery result.
- API introduction: findContours finds the contour/drawContours draws the contour
Sometimes, the concepts of contour and edge are very similar, and the contour of the surface on a single object is equivalent to its edge feature. But after multiple objects are superimposed, outlines and edges can no longer be compared in this way. (As shown in the picture above)
The outline is a topological map of the outline based on the edges, and then uses different topological algorithms to find and construct the outline. Therefore, the selected threshold value for edge extraction will affect the final contour discovery result.
After talking about the relationship and difference between edges and contours. So in OpenCV, how to realize the discovery and drawing of contours?
- Convert input image to grayscale image cvtColor
- Use Canny for edge extraction and convert binary images
- Discover contours using findContours
- Draw contours using drawContours
Here we first introduce the two APIs cv::findCountours and cv::drawContours
CV_EXPORTS_W void findContours(
InputArray image, // 输入图像，二值图，一般就是Canny的输出，8-bit
OutputArrayOfArrays contours, // 全部发现的轮廓对象，就是一个二维数组，图结构，往下细说
OutputArray hierarchy, // 轮廓图的拓扑结构，可选输出，最终的轮廓发现就是基于这个拓扑结构实现
int mode, // 寻找轮廓的模式，一般返回RETR_TREE树模式
int method, // 轮廓发现的方法，一般使用CHAIN_APPROX_SIMPLE简单方式
Point offset = Point() // 轮廓像素偏移，默认（0，0）没偏移
);
CV_EXPORTS_W void drawContours(
InputOutputArray image, // 绘制的目标图像
InputArrayOfArrays contours, // 全部轮廓对象，就是findContours的第二个输出参数
int contourIdx, // 轮廓索引号，contours的第一维索引
const Scalar& color, // 绘制颜色
int thickness = 1, // 绘制线宽
int lineType = LINE_8, // 绘制线类型
InputArray hierarchy = noArray(), // 拓扑结构图，findContours的第三个可选输出参数
int maxLevel = INT_MAX, // 最大层数，0只绘制当前的，1包含内部轮廓，2所有轮廓
Point offset = Point() // 轮廓偏移
);Next, let’s take a case code to explain how to understand the second and third parameters of findContours.
int main()
{
//读取测试图片
src = imread("F:\\other\\learncv\\bottle.png");
namedWindow(titleStr + "src", WINDOW_AUTOSIZE);
imshow(titleStr + "src", src);
//rgb转gray
cvtColor(src, gray, COLOR_BGR2GRAY);
namedWindow(titleStr + "circles", WINDOW_AUTOSIZE);
namedWindow(titleStr + "contours", WINDOW_AUTOSIZE);
createTrackbar("边缘检测阈值", titleStr + "src", &threshold_value, threshold_max, Callback_Contours);
waitKey(0);
return 0;
}
void Callback_Contours(int pos, void* userdata) {
Mat canny_img;
Canny(gray, canny_img, threshold_value, threshold_value * 2.0, 3, false);
vector<vector<Point>> contours;
vector<Vec4i> hierarchy;
findContours(canny_img, contours, hierarchy, RETR_TREE, CHAIN_APPROX_SIMPLE, Point(0, 0));
Mat dst1 = Mat::zeros(gray.size(), CV_8UC3);
Mat dst2 = Mat::zeros(gray.size(), CV_8UC3);
for (size_t i = 0; i < contours.size(); i++) {
drawContours(dst1, contours, i, colorWhite, 1, LINE_8, hierarchy, 0, Point(0, 0));
vector<Point> contourPoints = contours[i];
for (size_t j = 0; j < contourPoints.size(); j++) {
circle(dst2, contourPoints[j], 1, colorWhite);
}
}
imshow(titleStr + "contours", dst1);
imshow(titleStr + "circles", dst2);
}
The effect of running the above code is that the Canny edge threshold is between 100 and 200. Among them, I also drew Canny's second output parameter contours in the form of points, which corresponds to the far left side of the picture, the middle part is the contour drawn by drawContours, and the right side is the original picture. Zooming in, you can clearly observe the position of the lotus petals above the word "Lotus". It is obvious that the dots are intermittent with a large part of the blank space in the middle. After drawing the outline, they can be connected into a line . This is because drawContours will judge whether these belong to the same line segment based on the first dimension of the contours two-dimensional data. By debugging, you can know that the length of each layer of contours[i] is different.
As for the third parameter hierarchy, which is the contour topology relationship, the content output by this parameter has a great relationship with the fourth parameter mode, which is the mode for finding contours. Take a detailed look at the detailed analysis of the following student.
(12) Detailed explanation of hierarchy of findContours function_findcontours hierarchy_Heng Youcheng's Blog-CSDN Blog To obtain the outline of an object, it is generally best to grayscale the image first and then perform threshold processing, and then use it to detect the outline. _findcontours hierarchy https://blog.csdn.net/lx_ros/article/details/126258801
Ok，That’s All.