opencv-canny operator and edge detection

Edge detection

Our process of matting is actually the process of finding the edge of an instance (such as a person) in the image. Therefore, our edge detection is actually the process of detecting the edge of the instance in the image.

Now there is a question, how do we distinguish the margins?

We found that for each instance, the gap between its edge and the surrounding pixels is generally relatively large. Our matting is to distinguish the edges of instances based on the obvious pixel gaps.

Therefore, the basis of matting is the obvious change of image pixels. The edge of the image is generally the position where the pixels on the image have obvious changes.

Then we use a picture to represent the pixels of the image:

Image Pixels
We found that where the image pixels change significantly, the tilt of the image is very large, that is to say, the absolute value of the slope is very large. This is consistent with the principles of the several operators we talked about before, the Sobel operator is the position of calculation:

The definition of edge detection in the book is as follows:

Edge detection is a basic problem in image processing and computer vision. The purpose of edge detection is to identify points with obvious brightness changes in digital images. Significant changes in image attributes usually reflect important events and changes in attributes.

Edge detection operator

• Sobel
• Scharr
• Laplace
• Canny

Canny operator

The Canny edge detection operator is a multi-level edge detection algorithm developed by John F. Canny in 1986. It includes the following five steps:

• 1. Apply Gaussian filtering to smooth the image, the purpose is to remove noise

• 2. Gray conversion

• 3. Find the intensity gradients of the image, Sobel, Scharr

• 4. Apply non-maximum suppression technology to eliminate edge false detection (it was not originally but detected)

• 5. Apply the double threshold method to determine the possible (potential) boundary

• • 5.1. Use hysteresis technology to track boundaries

Introduce in turn

Image smoothing

Image smoothing is to make the pixel gap between images smaller, and to better extract edges with more obvious features, so the purpose of image smoothing is to remove part of the noise .

As for the image smoothing operation, let's think about it. Isn't it just to make the gap between the adjacent pixels smaller? Isn't that the effect of image blur? Therefore, the image smoothing operation is to perform the image blur operation, and the most commonly used blur operation is Gaussian blur.

Grayscale conversion

cv::cvtColor

Image gradient

The basic idea of ​​Canny's algorithm is to find the position of the strongest gray intensity change in an image . The so-called strongest change refers to the direction of the gradient. The gradient of each pixel in the smoothed image can be obtained by the Sobel operator (a convolution operation) (there is a packaged function in opencv, which can calculate the n-order derivative of each pixel in the image). First, the following kernels are used to obtain the gradients G_X and G_Y along the horizontal (x) and vertical (y) directions, respectively.

Sobel and Scharr

Non-maximum suppression

In order to eliminate edge detection

The purpose of this step is to sharpen the blurred boundary. In layman's terms, the maximum value of the gradient intensity on each pixel is retained , and other values ​​are deleted. For each pixel, perform the following operations:

• a) Approximate its gradient direction to one of the following values:

                       （0,45,90,135,180,225,270,315）（即上下左右和45度方向）
  

• b) Compare the pixel point with the gradient intensity of the pixel point in the positive and negative direction of its gradient direction

• c) If the gradient intensity of the pixel is the largest, keep it, otherwise suppress (delete, set to 0)

In order to better explain this concept, look at the following figure:

the numbers in the figure represent the gradient intensity of the pixel, and the arrow direction represents the gradient direction.
Take the third pixel in the second row as an example. Since the gradient direction is upward, the intensity of this point (7) is compared with the intensities of the two pixels above and below (5 and 4). Since this point has the highest intensity, it is retained.

Here is another example to explain:

In the previous step of the gradient calculation, the gradients in the x and y directions were obtained. Then we can get the direction (angle) of the gradient based on this gradient.

For example, the gradient in the horizontal direction changes the most, then we keep the direction perpendicular to it (the farthest distance), and remove both the left and right sides.

which is:

• The two areas of 0-22.5 and 157.5-180 are regarded as the horizontal direction, then we compare it with its upper and lower pixel values. If the upper and lower pixel values ​​are less than this pixel, then both the upper and lower pixels are discarded. The current pixel Reserved; if the upper and lower pixel values ​​are greater than this pixel, then the current pixel is discarded

• In the interval of 67.5-112.5, it is regarded as horizontal, and the current pixel is compared with the adjacent pixels on the left and right.

• In the interval of 112.5-157.5, it is regarded as the upper left and lower right direction, and the current pixel is compared with the adjacent pixel on the lower left and upper right

• In the interval of 22.5-67.5, it is regarded as the lower left and upper right direction, and the current pixel is compared with the adjacent pixels on the upper left and lower right

Double-threshold possible edge

After non-maximum suppression, there are still many noise points in the image.

In the Canny algorithm, a technique called double threshold is applied. That is, set an upper threshold and a lower threshold (usually designated by humans in opencv)

• If the pixel in the image is larger than the upper threshold of the threshold, it must be considered as a boundary (called a strong edge)
• If it is less than the lower bound of the threshold, it must not be a boundary
• The one between the two is considered as a candidate (called weak edge), and it is accepted only when it is connected to a pixel above the threshold, otherwise it is discarded. This step is called boundary tracking in opencv

The upper threshold and the lower threshold are also called high threshold and low threshold.

The Canny algorithm suggests a high threshold: the low threshold is between 2:1 and 3:1.

The picture below is the result of processing a test pattern and photo with high: low at 5:1 and 3:2.

Boundary tracking

A higher brightness gradient is more likely to be an edge, and a brightness below the low threshold must not be an edge.
But there is no exact value to define whether the brightness gradient between the high and low thresholds is an edge, so Canny uses a hysteresis threshold.

Hysteresis thresholding requires two thresholds, namely a high threshold and a low threshold.
It is assumed that the important edges in the image are all continuous curves, so that we can track the blurred part of a given curve and avoid the noise pixels that do not make up the curve as edges. So we start with a larger threshold, which will identify the true edges that we are more confident about. Using the direction information derived earlier, we start from these true edges and track the entire edge in the image. When tracking, we use a smaller threshold so that we can track the fuzzy part of the curve until we get back to the starting point.

which is:

It is accepted when the edge between the high and low threshold is connected to a pixel above the threshold

API

void Canny( 
    InputArray src, 
    OutputArray edges, 
    double threshold1, 
    double threshold2,
    int apertureSize = 3, 
    bool L2gradient = false
);

The meaning is as follows

• (1) src of InputArray type, input image.
  Must be a single-channel image.

• (2) Edges of OutputArray type, output edge image; single-channel 8-bit image, the size of which is the same as the image.
  The output image must be a grayscale image (actually a Boolean image), generally let its background be black

• (3) Double type threshold1, the first threshold of the hysteresis process—low threshold.
  Low threshold, usually 1/2 or 1/3 of the high threshold

• (4) The double type threshold2, the second threshold of the hysteresis process-the high threshold.
  High threshold

• (5) Int type apertureSize, Sobel operator Size, usually 3x3, the value is 3.

• (6) L2gradient of bool type, a flag indicating whether the more accurate L2 should be used to calculate the image gradient amplitude (L2gradient=true), or whether the default L1 (L2gradient=false) should be used. (Note: here L2 is the root of the square sum, L1 is the sum of absolute values, as shown in the figure below )
  Generally , L1 is used instead of L2, and the default is false

Code

#include<iostream>
#include<opencv2/opencv.hpp>

using namespace std;
using namespace cv;

Mat src, gray, dst;
Mat output;
int t1_value = 50;
int max_value = 255;

void Canny_demo(int,void*)
{
    
    

	blur(gray, gray, Size(3,3), Point(-1, -1), BORDER_DEFAULT);
	Canny(gray, output, t1_value, t1_value*2, 3, false);
	
	dst.create(src.size(), src.type());//将二值灰度图彩色化
	src.copyTo(dst, output);//将源图像像素拷贝到二值图为白色的对应像素
	imshow("bin img", output);
}

int main()
{
    
    


	namedWindow("input img", CV_WINDOW_AUTOSIZE);
	namedWindow("output img", CV_WINDOW_AUTOSIZE);

	src = imread("C:/Users/86176/Pictures/pics/lena(1).tiff");
	if (!src.data)
	{
    
    
		printf("fail to load img\n");
		return -1;
	}
	imshow("input img", src);

	cvtColor(src, gray, CV_BGR2GRAY);
	createTrackbar("Threshold Value:", "output img", &t1_value, max_value, Canny_demo);
	Canny_demo(0,0);

	imshow("output img", dst);

	waitKey(0);
	return 0;
}

result

The lower the high threshold, the more details are retained