Image processing and computer vision--Chapter 5-Image segmentation-Canny operator

1. Classification of edge detection operators

(1)一阶导数的边缘检测算子:
  通过模板作为核与图像的每个像素点做卷积和运算，然后选取合适的阈值来提取图像的边缘。常见的有Roberts算子、Sobel算子和Prewitt算子。
(2)二阶导数的边缘算子:
  依据于二阶导数过零点，常见的有Laplacian 算子，此类算子对噪声敏感。
(3)Canny算子:
  前面两类均是通过微分算子来检测图像边缘，还有一种就是Canny算子，其是在满足一定约束条件下推导出来的边缘检测最优化算子。

2.Canny operator core theory

2.1.A brief introduction to the Canny operator

The Canny operator is a very commonly used edge detection operator. Its rendering is as follows:

2.2. Canny operator edge detection index

The Canny operator is implemented based on edge detection, so the indicators of edge detection are as follows:
(1) Good signal-to-noise ratio, that is, the probability of determining non-edge points as edge points is low.
(2) High positioning, the detected edge should be at the center of the actual edge.
(3) There is only a unique response to a single edge, that is, false edges must be maximally suppressed.

2.3.Basic principles of Canny operator

1.2个条件:一能有效抑制噪声，二能精确定位边缘位置
2.平滑后求导，用高斯滤波器进行平滑
3.用一阶偏导有限差分计算梯度幅值与方向
3.对梯度幅值进行非极大值抑制
4.用双闯值检测与连接边缘

3. Canny operator processing flow

The content of the third part is quoted from: https://blog.csdn.net/zaishuiyifangxym/article/details/90142702
My job is to learn through his blog. This blogger’s CV column is very good.
If you don’t understand anything in this blog, you can check out his column. The content is very comprehensive and you should be able to get a better answer.

3.1. Gaussian filtering to remove noise

According to the first principle of the above algorithm, we need to denoise the impact of the image.
Edge detection in images susceptible to noise. Therefore, denoising is usually required before edge detection. Usually, Gaussian filtering is used to remove noise. Gaussian filtering is a commonly used denoising function. Its 5x5 template formula corresponds to the following: 1 273 × [ 1 4 7 4 1 4 16 26 16 4 7 26 41 26 7
$$\frac{1}{273}\times \left[\begin{array}{ccccc}1& 4& 7& 4& 1\\ 4& 16& 26& 16& 4\\ 7& 26& 41& 26& 7\\ 4& 16& 26& 16& 4\\ 1& 4& 7& 4& 1\end{array}\right]$$

In the Opencv library, the Gaussian filter function is as follows:

img = cv2.GaussianBlur(src, ksize, sigmaX)
其中，参数：src 表示原始图像；ksize 表示核大小,而且ksize只能是奇数；sigmaX 表示X方向方差。

3.2. Image gradient search

In this part of the content, we need to calculate and find the gradient amplitude and direction, and find the gradient of the image. We need to first apply the convolution template to the x and y directions respectively, and then calculate the gradient amplitude and direction. The formula is as follows:
$$\begin{array}{rl}Convolution:d_x& =\left[\begin{array}{ccc}-1& 0& 1\\ -2& 0& 2\\ -1& 0& 1\end{array}\right]d_y=\left[\begin{array}{ccc}-1& -2& -1\\ 0& 0& 0\\ 1& 2& 1\end{array}\right]\\ & \\ Amplitude:S& =\sqrt{d_x^2+d_y^2}\\ & \\ direction:& =\arctan (\frac{d_y}{d_x})\end{array}$$
The amplitude and direction obtained after calculation are shown in the figure:
              

3.3. Non-maximum suppression processing

  For each pixel, it does the following: applies non-maximum suppression technology to filter out non-edge pixels and make blurry boundaries clear. This process retains the maximum value of the gradient intensity at each pixel and filters out other values.

Step1：将其梯度方向近似为以下值中的一个，包括0、45、90、135、180、225、270和315，即表示上下左右和45度方向。
Step2：比较该像素点和其梯度正负方向的像素点的梯度强度，如果该像素点梯度强度最大则保留，否则抑制（删除，即置为0）。

The processing methods of Step1 and Step2 are as shown in the figure:

3.4.Double threshold boundary processing

  The principle of dual-threshold boundary processing is also relatively simple. We apply two thresholds $T_{low}$和$T_{high}$. Make the gradient value less than $T_{low}$The gray value of the pixel is set to 0, and image 1 is obtained. Then make the gradient value greater than $T_{high}$The grayscale value of the pixel is set to 0, and image 2 is obtained.
 Since the threshold value of image 2 is relatively high, most of the noise is removed, but useful edge information is also lost. Image 1 has a lower threshold and retains more information. We can use image 2 as a basis and image 1 as a supplement to connect the edges of the image.
     

3.5. Boundary lag technology tracking

Research on boundary lag technology tracking methods comes from: https://blog.csdn.net/qq_44736333/article/details/109152380

 We assume two types of edges: among edge points after non-maximum suppression, those with gradient values ​​exceeding T1 are called strong edges, those with gradient values ​​less than T1 and greater than T2 are called weak edges, and those with gradients less than T2 are not edges.
 What is certain is that strong edges must be edge points, so T1 must be set high enough to require the gradient value of the pixel to be large enough (the change is drastic enough), while weak edges may be edges or noise. How to judge? Woolen cloth?
 When there are strong edge points in the eight neighborhoods around a weak edge, the weak edge point is turned into a strong edge point to supplement the strong edge.
 In practice, people find that the ratio of T1:T2=2:1 is better. T1 can be specified manually, or an algorithm can be designed to adaptively specify it. For example, the dividing line of the first 30% of the gradient histogram is defined as T1, which I implemented. It is an artificially specified threshold.

 In the picture above, the white square represents the existence of an edge (including strong and weak edges). Each pixel in the weak edge is traversed. If there is a pixel corresponding to a strong edge in the eight neighborhoods of the pixel, this weak edge pixel is classified as a real edge. (To understand from a visual perspective, there is an uncertain edge. If there is a real edge next to this uncertain edge, then this edge will be classified as a real edge, otherwise it will be deleted)

3.6. Characteristics of Canny operator edge detection

1.检出准则: 该准则要求不应丢失重要的边缘，而且不应输出噪声性的多余边缘。
2.定位准则: 该准则要求检测到的边缘位置与实际边缘位置之间的距离为最小。
3.单一响应准则: 该准则在一定程度上多涵盖为检出准则，它要求尽可能减少对于同一边缘出现多个响应的情况。

4. Canny operator algorithm code

import cv2
import numpy as np
import matplotlib.pyplot as plt
# 支持中文
plt.rcParams['font.sans-serif'] = ['SimHei'] # 用来正常显示中文标签
plt.rcParams['axes.unicode_minus'] = False # 用来正常显示负号

def All_Canny(img):
    grayImage = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)#图像灰度化处理
    gaussian = cv2.GaussianBlur(grayImage, (5, 5), 0)# 高斯滤波降噪
    Canny = cv2.Canny(gaussian, 50, 150)# Canny算子
    return grayImage, gaussian,Canny

# 读取图像
img = cv2.imread('lena.jpg')#imread出来的是BRG图像
lenna_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)#转化为RGB图像
# 显示图形
Figure=All_Canny(img)#获取Canny检测之后的图片

#图像可视化处理
titles = [u'原始图像', u'灰度处理图',u'高斯去噪处理图', u'Canny算子c处理图']
images = [lenna_img ,Figure[0],Figure[1], Figure[2]]
#subplot绘制
#第一张图
plt.subplot(2, 2, 1)
plt.imshow(images[0], 'gray'),plt.title(titles[0])
plt.xticks([]),plt.yticks([])
#第二张图
plt.subplot(2, 2, 2)
plt.imshow(images[1], 'gray'),plt.title(titles[1])
plt.xticks([]),plt.yticks([])
#第三张图
plt.subplot(2, 2, 3)
plt.imshow(images[2], 'gray'),plt.title(titles[2])
plt.xticks([]),plt.yticks([])
#第四张图
plt.subplot(2, 2, 4)
plt.imshow(images[3], 'gray'),plt.title(titles[3])
plt.xticks([]),plt.yticks([])

plt.savefig(r"D:\HuaweiMoveData\Users\27182\Desktop\py\Canny.png",dpi=1000)
plt.show()

5. Canny operator effect display

6. Reference articles and acknowledgments

本章内容的完成离不开以下大佬文章的启发和帮助，在这里列出名单，如果对于内容还有不懂的，可以移步对应的文章进行进一步的理解分析。
1.Canny边缘检测算子流程:https://blog.csdn.net/zaishuiyifangxym/article/details/90142702
2.边界滞后技术跟踪算法:https://blog.csdn.net/qq_44736333/article/details/109152380
我的工作就是根据两位大佬的blog来进行学习,这两位博主的cv的专栏写的非常好。
如果大家这这篇blog中有什么不明白的可以去他们俩个的专栏里面看看，内容非常全面，应该能够有比较好的解答。
在文章的最后再次表达由衷的感谢！！