Image edge information is mainly concentrated in high frequency bands. Generally speaking, image sharpening or edge detection is essentially high-frequency filtering. We know that the differential operation is to find the rate of change of the signal, which has the effect of strengthening the high frequency components. In spatial operations, the sharpening of the image is the calculation of differentiation. Due to the discrete signal of the digital image, the differential operation becomes the calculation of the difference or gradient. There are a variety of edge detection (gradient) operators in image processing. Commonly used include ordinary first-order difference, Robert operator (cross-difference), Sobel operator, etc., which are based on finding gradient strength. Laplace operator (second-order difference) is based on zero-crossing detection. By calculating the gradient and setting the threshold, the edge image is obtained.
What is the edge? The part where the brightness of the local area of the image changes significantly, for the gray image, that is, the gray value has an obvious change, from a sharp change in a gray value in a small buffer area to a large difference in gray value. The gray value. While improving the sensitivity to the edge of the scene, the method that can suppress noise is a good edge extraction method.
The specific algorithm steps of Canny operator to find edge points are as follows:
1. Smooth the image with Gaussian filter.
2. Calculate the gradient magnitude and direction using first-order partial derivative finite difference.
3. Non-maximum suppression of gradient amplitude.
4. Use dual threshold algorithm to detect and connect edges.
Step 1: Grayscale
Turn the color image into a grayscale image. This part is the grayscale image that is usually processed according to the Canny algorithm. If the color image is obtained, the grayscale must be performed first. Taking the color image in RGB format as an example, the formula for grayscale is usually:
Gray=0.299R+0.587G+0.114B;
Step 2: Gaussian filtering
For image Gaussian filtering, the realization of image Gaussian filtering can be implemented with two one-dimensional Gaussian kernels twice weighted respectively, that is, first one-dimensional X-direction convolution, and the obtained result is then one-dimensional Y-direction convolution. Of course, it can also be implemented directly through a convolution of a two-dimensional Gaussian kernel. That is, the two-dimensional convolution template. Due to the limited level, only how to calculate the two-dimensional convolution template.
First, the one-dimensional Gaussian function:
Two-dimensional Gaussian function:
The Gaussian coefficient of each point in the template can be calculated by the above formula. Is the resulting template the final template? The answer is no, it needs to be normalized, that is, the coefficient of each point must be divided by the sum of all coefficients, so that it is the final two-dimensional Gaussian template.
There is a small knowledge point in this. To calculate the above coefficients, you need to know the standard deviation σ (sigma) of the Gaussian function, and you also need to know whether to choose a 3*3 or 5*5 template, that is, how big the template is and what is actually used At that time, the two are related. According to the knowledge of mathematical statistics, the Gaussian distribution is characterized by the probability that the value is distributed in (μ-3σ, μ+3σ) is 0.9974, that is, the size of the template is actually as large as 6σ OK, but 6σ may not be an odd number, because we must ensure that there is a core. Therefore, the size of the template window is generally 1+2*ceil(3*nSigma). ceil is a round-up function, for example, ceil(0.6)=1.
Calculate the template, that is, direct convolution is OK. Convolution means that the template size area near the point in the image is multiplied by the Gaussian template area, and the result obtained is the result of the point convolution. The core meaning of convolution is to obtain the characteristics of the original image like template.
Step 3: Calculate the gradient value and direction
The edges of the image can point to different directions, so the classic Canny algorithm uses four gradient operators to calculate the horizontal, vertical and diagonal gradients respectively. But usually four gradient operators are not used to calculate the four directions separately. Commonly used edge difference operators (such as Rober, Prewitt, Sobel) calculate the horizontal and vertical difference Gx and Gy. In this way, the gradient mode and direction can be calculated as follows:
The gradient angle θ ranges from radians -π to π, and then approximates it to four directions, representing horizontal, vertical and two diagonal directions (0°, 45°, 90°, 135°). It can be divided by ±iπ/8 (i=1,3,5,7), and the gradient angle falling in each area is given a specific value, which represents one of the four directions.
Here, the Sobel operator is selected to calculate the gradient. Compared with other edge operators, the edge obtained by the Sobel operator is thick and bright.
Sobel operator, no threshold
Step 4: Non-maximum suppression
Non-maximum suppression is an important step in edge detection. In a popular sense, it refers to finding the local maximum of a pixel. Along the gradient direction, the gradient value before and after it is compared. See below.
The left and right images in the above figure: g1, g2, g3, and g4 all represent pixel points. Obviously they are 4 of the eight areas of c. Point c in the left image is the point we need to judge, and the blue line is its The gradient direction, that is to say, if c is a local maximum, its gradient amplitude M needs to be greater than the intersection of the straight line and g1g2 and g2g3, and the gradient amplitude at dtmp1 and dtmp2. But dtmp1 and dtmp2 are not whole pixels, but sub-pixels, that is, the coordinates are floating-point, so how to find their gradient amplitude? Linear interpolation, for example, dtmp1 is between g1 and g2, and the amplitudes of g1 and g2 are known. As long as we know the ratio of dtmp1 between g1 and g2, we can get its gradient amplitude, and the ratio can depend on the angle The calculated angle is the direction of the gradient.
Write a linear interpolation formula: set the amplitude of g1 M(g1) and the amplitude of g2 M(g2), then dtmp1 can be obtained:
M(dtmp1)=w*M(g2)+(1-w)*M(g1)
Where w=distance(dtmp1,g2)/distance(g1,g2)
distance(g1,g2) represents the distance between two points. In fact, w is a scale factor, which can be obtained through the gradient direction (tangent and cotangent of the argument).
The 4 straight lines in the figure on the right represent 4 different situations. The situations are different. The 4 coordinates in the eight fields where g1, g2, g3, and g4 represent c will be different, but the principle of linear interpolation is the same.
The figure below is the result of non-maximum suppression. It can be seen that the edge width has been greatly reduced. However, because no threshold is applied, this image still contains a large number of points with small gradient modulus, which is the very dark place in the image. Next, the threshold is about to play.
Step 5: Selection of dual thresholds
General edge detection algorithms use a threshold to filter out small gradient values caused by noise or color changes, while retaining large gradient values. The Canny algorithm uses dual thresholds, that is, a high threshold and a low threshold to distinguish edge pixels. If the edge pixel point gradient value is greater than the high threshold, it is considered a strong edge point. If the edge gradient value is less than the high threshold and greater than the low threshold, it is marked as a weak edge point. Points smaller than the low threshold are suppressed.
Step 6: Lag boundary tracking
Strong edge points can be considered true edges. Weak edge points may be true edges, or they may be caused by noise or color changes. In order to obtain accurate results, the weak edge points caused by the latter should be removed. It is generally believed that weak edge points and strong edge points caused by real edges are connected, but weak edge points caused by noise are not. The so-called hysteresis boundary tracking algorithm checks the 8-connected domain pixels of a weak edge point. As long as there is a strong edge point, then the weak edge point is considered to be a true edge.
This algorithm searches for all connected weak edges. If any point of a connected weak edge is connected to a strong edge point, the weak edge is retained, otherwise the weak edge is suppressed. You can use breadth-first or depth-first algorithms when searching. Here I have implemented the easiest depth-first algorithm. The depth-first algorithm for connecting one edge at a time is as follows:
- Prepare a stack s and a queue q, and set the connected indicator variable connected to false. Starting from the first point of the image, go to 2.
- If this point is a weak boundary point and has not been marked, mark it and put it in the stack s as the first element, and put it in the queue q that records the connected curve, and enter 3. If this point is not a weak boundary or has been marked, go to the next point in the image and repeat 2.
- Take an element from the stack s and find its 8-pixel area. If a field pixel is a weak boundary and has not been marked, mark the field pixel and add it to stack s and add queue q at the same time. At the same time, find the strong boundary map corresponding to the field. If a pixel is a strong boundary, it means that the weak boundary curve is connected with the strong boundary. Set connected to true. Repeat 3 until there are no more elements in the stack. If connected is false, each element is taken from the queue q in turn, and the mark is cleared. If connected is true, keep the mark.
- Clear the queue q, set connected to false, and move to the next point of the image, to 2.
Step 7: View the results
The following is the gradient modulus and binarization graph of Canny edge detection calculated for Lena graph, Gaussian radius 2, high threshold 100, low threshold 50.
For comparison, the following is the result of calculating the original image with the first-order difference and the Sobel operator, the threshold is 100. Since the gradient value of the first-order difference is relatively small, I magnified the gradient value of the first-order difference by a certain factor to keep it at the same level as Sobel's gradient value.
Obviously, the effect of Canny edge detection is very significant. Compared with the ordinary gradient algorithm, the false edge caused by noise is greatly suppressed, and the edge is refined, which is easy for subsequent processing. For images with low contrast, the Canny algorithm can also have good results by adjusting the parameters.
public Bitmap CANNY(Bitmap oldBitmap, int Width, int Height) //定义canny函数
{
Bitmap newBitmap = new Bitmap(Width, Height);
cannyClass2 cannyclass3 = new cannyClass2(); //Generate an instance of the class
float gradientMax = 0;
float[,] gradient = new float[Width, Height];
byte[,] degree = new byte[Width, Height];
int[,] srcBytes = new int[Width, Height];
int highThreshould = 24; //high threshold
int lowThreshould = 10; //low threshold
int ret = 0;
for (int j = 0; j < Height; j++)
{
for (int i = 0; i < Width; i++)
{
srcBytes[i, j] = oldBitmap.GetPixel(i, j).R;
}
}
cannyclass3.GetGradientDegree(srcBytes, ref gradient, ref degree, ref gradientMax, Width, Height); //Call the gradient function in the class
cannyclass3.NonMaxMini(gradient, ref srcBytes, gradientMax, Width, Height, degree); //Non-maximum suppression
cannyclass3.TwoThreshouldJudge(highThreshould, lowThreshould, ref srcBytes, Width, Height); //Double threshold edge judgment
for (int j = 0; j < Height; j++)
{
for (int i = 0; i < Width; i++)
{
ret = srcBytes[i, j];
newBitmap.SetPixel(i, j, Color.FromArgb(ret, ret, ret));
}
}
return newBitmap;
}
Gaussian filtering
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Drawing;
namespace WindowsFormsApplication1
{
class gsClass1
{
public int BlurRadius = 1; //高斯函数宽度
public List<double> BlurArray = new List<double>();
public int MaxWidth;
public int MaxHeight;
private double GetWeighing(int x, int y) //定义高斯公式
{
double q = (BlurRadius * 2 + 1) / 2;
return 1 / (2 * Math.PI * Math.Pow(q, 2)) * Math.Exp(-(x * x + y * y) / (2 * q * q));
}
public Color GetBlurColor(Bitmap oldBitmap, int x, int y)
{
double r = 0, g = 0, b = 0;
int index = 0;
for (var t = y - this.BlurRadius; t <= y + this.BlurRadius; t++)
{
for (var l = x - this.BlurRadius; l <= x + this.BlurRadius; l++)
{
var color = GetDefautColor(oldBitmap, l, t);
var weighValue = BlurArray[index];
r += color.R * weighValue;
g += color.G * weighValue;
b += color.B * weighValue;
index++;
}
}
return Color.FromArgb((byte)r, (byte)g, (byte)b);
}
public Color GetDefautColor(Bitmap oldBitmap, int x, int y)
{
if (x < 0 && y < 0)
return oldBitmap.GetPixel(0, 0);
else if (x < 0)
return oldBitmap.GetPixel(0, Math.Min(MaxHeight, y));
else if (y < 0)
return oldBitmap.GetPixel(Math.Min(MaxWidth, x), 0);
else
return oldBitmap.GetPixel(Math.Min(MaxWidth, x), Math.Min(MaxHeight, y));
}
public void SetBlurArray()
{
int blur = BlurRadius;
double sum = 0;
for (var y = blur; y >= blur * -1; y--)
{
for (var x = blur * -1; x <= blur; x++)
{
var d = GetWeighing(x, y);
BlurArray.Add(d);
sum += d;
}
}
for (var i = 0; i < BlurArray.Count; i++)
BlurArray[i] = BlurArray[i] / sum;
//sum = 0;
//foreach (var item in this.BlurArray)
// sum += item;
}
}
}
CANNY
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Drawing;
namespace WindowsFormsApplication1
{
class cannyClass2
{
public void NonMaxMini(float[,] gradient, ref int[,] srcBytes, float GradientMax, int x, int y, byte[,] degree) //non-maximum suppression
{
float leftPixel = 0, rightPixel = 0;
for (int j = 1; j < y - 1; j++)
{
for (int i = 1; i < x - 1; i++)
{
switch (degree[i, j])
{
case 0:
leftPixel = gradient[i - 1, j];
rightPixel = gradient[i + 1, j];
break;
case 45:
leftPixel = gradient[i - 1, j + 1];
rightPixel = gradient[i + 1, j - 1];
break;
case 90:
leftPixel = gradient[i, j + 1];
rightPixel = gradient[i, j - 1];
break;
case 135:
leftPixel = gradient[i + 1, j + 1];
rightPixel = gradient[i - 1, j - 1];
break;
default:
break;
}
if ((gradient[i, j] < leftPixel) || (gradient[i, j] < rightPixel))
{
srcBytes[i, j] = 0;
}
else
{
srcBytes[i, j] = (int)(255 * gradient[i, j] / GradientMax);
}
}
}
}
//Double threshold edge judgment
public void TwoThreshouldJudge(int highThreshold, int lowThreshould, ref int[,] srcBytes, int x, int y)
{
for (int j = 1; j < y - 1; j++)
{
for (int i = 1; i < x - 1; i++)
{
if (srcBytes[i, j] > highThreshold)
{
srcBytes[i, j] = 255;
}
else if (srcBytes[i, j] < lowThreshould)
{
srcBytes[i, j] = 0;
}
else
{
if (srcBytes[i - 1, j - 1] < highThreshold && srcBytes[i, j - 1] < highThreshold && srcBytes[i + 1, j - 1] < highThreshold && srcBytes[i - 1, j] < highThreshold
&& srcBytes[i + 1, j] < highThreshold && srcBytes[i - 1, j + 1] < highThreshold && srcBytes[i, j + 1] < highThreshold && srcBytes[i + 1, j + 1] < highThreshold)
{
srcBytes[i, j] = 0;
}
else
srcBytes[i, j] = 255;
}
}
}
}
public void GetGradientDegree(int[,] srcBytes, ref float[,] gradient, ref byte[,] degree, ref float GradientMax, int x, int y) //梯度公式
{
gradient = new float[x, y];
degree = new byte[x, y];
int gx, gy;
int temp;
double div;
for (int j = 1; j < y - 1; j++)
{
for (int i = 1; i < x - 1; i++)
{
gx = srcBytes[i + 1, j - 1] + 2 * srcBytes[i + 1, j] + srcBytes[i + 1, j + 1] - srcBytes[i - 1, j - 1] - 2 * srcBytes[i - 1, j] - srcBytes[i - 1, j + 1];
gy = srcBytes[i - 1, j - 1] + 2 * srcBytes[i, j - 1] + srcBytes[i + 1, j - 1] - srcBytes[i - 1, j + 1] - 2 * srcBytes[i, j + 1] - srcBytes[i + 1, j + 1];
gradient[i, j] = (float)Math.Sqrt((double)(gx * gx + gy * gy)); //Gradient formula
if (GradientMax < gradient[i, j])
{
GradientMax = gradient[i, j];
}
if (gx == 0)
{
temp = (gy == 0) ? 0 : 90;
}
else
{
div = (double)gy / (double)gx;
if (div < 0)
{
temp = (int)(180 - Math.Atan(-div) * 180 / Math.PI);
}
else
{
temp = (int)(Math.Atan(div) * 180 / Math.PI);
}
if (temp < 22.5)
{
temp = 0;
}
else if (temp < 67.5)
{
temp = 45;
}
else if (temp < 112.5)
{
temp = 90;
}
else if (temp < 157.5)
{
temp = 135;
}
else
temp = 0;
}
degree[i, j] = (byte)temp;
}
}
}
}
}
