Table of contents
1. Problem description
At present, many key mechanical parts are steel welded structures, and steel welded structures are prone to defects such as cracks, missing welds, and irregular appearance of welds, so the quality inspection of welds is particularly important. The edge of the weld is the most important feature of the weld image. The classic edge extraction algorithm realizes edge extraction by considering the gray level change between connected pixels and using the change rule of the first or second order derivative of the edge adjacency. In some commonly used edge detection operators, Sobel often forms an unclosed area, and other operators such as Laplace operator usually produce heavy responses. ** This paper uses T-shaped welding seam images for analysis, and discusses the method of weld edge detection based on morphological processing, which has a large signal-to-noise ratio and high precision. **The algorithm first uses image preprocessing techniques such as median filtering, white balance processing, and image normalization processing to correct the collected image, and then uses the morphological processing algorithm to extract the binary image of the weld. This algorithm not only effectively reduces noise , and ensure that the useful information of the image is not lost. The weld image is as follows:
The code is as follows:
clear
close all;
ps=imread('1.jpg');
subplot(121),imshow(ps)
background=imopen(ps,strel('disk',4));
% imshow(background);
subplot(122),surf(double(background(1:4:end,1:4:end))),zlim([0 256]);
set(gca,'Ydir','reverse');
2. Image preprocessing
2.1 Median filter denoising
Median filtering is a neighborhood operation, similar to convolution. The median value of the signal is the median value arranged in order of signal value. Using median filtering can better eliminate pulse interference and maintain signal edges. code show as below:
%% 读取焊缝图像
clc,clear,close all
obj=imread('1.jpg');
r=obj(:,:,1);g=obj(:,:,2);b=obj(:,:,3);
%% 去噪中值滤波
objn=imnoise(g,'salt & pepper',0.04);%加入少许椒盐噪声
K1=medfilt2(objn,[3,3]);%3x3模板中值滤波
K2=medfilt2(objn,[5,5]);%5x5模板中值滤波
figure('color',[1,1,1])
subplot(121),imshow(objn);xlabel('加噪图像')
subplot(122),imshow(K1);xlabel('去噪图像')
operation result:
2.2 White balance processing
Due to the large amount of noise in the captured image, the colors of all objects in the image often deviate from their original true colors, and the white balance method is used to deal with image distortion, thereby correcting the color of the image with colored chips to obtain an image of normal color. code show as below:
clc,clear,close all
img=imread('1.jpg');
subplot(121),imshow(img),xlabel('原始图像')
img_1=img(:,:,1);
img_2=img(:,:,2);
img_3=img(:,:,3);
Y=0.299*img_1+0.587*img_2+0.114*img_3; % 白平衡系数
[m,n]=size(Y);
k=Y(1,1);
for i=1:m
for j=1:n
if Y(i,j)>=k
k=Y(i,j);
k1=i;
k2=j;
end
end
end
[m1,n1]=find(Y==k);
Rave=sum(sum(img_1));
Rave=Rave/(m*n);
Gave=sum(sum(img_2));
Gave=Gave/(m*n);
Bave=sum(sum(img_3));
Bave=Bave/(m*n);
Kave=(Rave+Gave+Bave)/3;
img__1=(Kave/Rave)*img_1;
img__2=(Kave/Gave)*img_2;
img__3=(Kave/Bave)*img_3;
imgysw=cat(3,img__1,img__2,img__3);
subplot(122),imshow(imgysw),xlabel('白平衡处理结果')
operation result:
3. Weld edge detection
Image edge is a collection of all pixels with discontinuous or sharp grayscale changes in the image, which concentrates most of the information of the image and is one of the most basic features of the image. Edge detection is a key step in the subsequent image analysis fields such as image segmentation, feature extraction and recognition, and plays a very important role in engineering applications. The traditional detection method determines the edge by calculating the first-order or second-order differential of each pixel of the image. The peak point of the first-order differential of the image or the zero-crossing point of the second-order differential corresponds to the edge pixel of the image. The more common detection operators are: Sobel , Prewitt and Canny operators.
3.1 Sobel operator edge detection
The Sobel operator is to weight the gray value difference of the upper, lower, left, and right neighborhoods of each pixel in the image, and reach the mechanism at the edge to detect the edge. It is defined as:
S x = [ f ( x + 1 , y − 1 ) + 2 f ( x + 1 , y ) + f ( x + 1 , y + 1 ) ] − [ f ( x − 1 , y − 1 ) + 2 f ( x − 1 , y ) + f ( x − 1 , y + 1 ) ] {S_x} = \left[ {f\left( {x + 1{\rm{,}}y{\ rm{ - }}1} \right) + 2f\left( {x + 1{\rm{,}}y} \right) + f\left( {x + 1{\rm{,}}y + 1 } \right)} \right]{\rm{ - }}\left[ {f\left( {x{\rm{ - }}1{\rm{,}}y{\rm{ - }}1} \right) + 2f\left( {x{\rm{ - }}1{\rm{,}}y} \right) + f\left( {x{\rm{ - }}1{\rm{, }}y + 1} \right)} \right]Sx=[f(x+1,y−1)+2 f(x+1,y)+f(x+1,y+1)]−[f(x−1,y−1)+2 f(x−1,y)+f(x−1,y+1)]
S y = [ f ( x − 1 , y + 1 ) + 2 f ( x , y + 1 ) + f ( x + 1 , y + 1 ) ] − [ f ( x − 1 , y − 1 ) + 2 f ( x , y − 1 ) + f ( x + 1 , y − 1 ) ] {S_y} = \left[ {f\left( {x{\rm{ - }}1{\rm{,}}y + 1} \right) + 2f\left( {x{\rm{,}}y + 1} \right) + f\left( {x + 1{\rm{,}}y + 1} \right)} \right]{\rm{ - }}\left[ {f\left( {x{\rm{ - }}1{\rm{,}}y{\rm{ - }}1} \right) + 2f\left( {x{\rm{,}}y{\rm{ - }}1} \right) + f\left( {x + 1{\rm{,}}y{\rm{ - }}1} \right)} \right] Sy=[f(x−1,y+1)+2 f(x,y+1)+f(x+1,y+1)]−[f(x−1,y−1)+2 f(x,y−1)+f(x+1 , y − 1 ) ]
Each pixel in the image is convolved with the following two kernels, one kernel has the greatest influence on the vertical edge, and the other kernel has the greatest influence on the horizontal edge, and the maximum value of the two convolutions is used as this pixel output value. Sobel operator convolution template is:
Sobel operator code is as follows:
%% Sobel
clc,clear,close all
I=imread('1.jpg'); % 读入图像
r=I(:,:,1);g=I(:,:,2);b=I(:,:,3);
nI=size(r);
im = single(I) / 255;
yfilter = fspecial('sobel'); % sobel
xfilter = yfilter';
rx = imfilter(im(:,:,1), xfilter);
gx = imfilter(im(:,:,2), xfilter);
bx = imfilter(im(:,:,3), xfilter);
ry = imfilter(im(:,:,1), yfilter);
gy = imfilter(im(:,:,2), yfilter);
by = imfilter(im(:,:,3), yfilter);
Jx = rx.^2 + gx.^2 + bx.^2;
Jy = ry.^2 + gy.^2 + by.^2;
Jxy = rx.*ry + gx.*gy + bx.*by;
D = sqrt(abs(Jx.^2 - 2*Jx.*Jy + Jy.^2 + 4*Jxy.^2)); % 2x2 matrix J'*J的第一个特征值
e1 = (Jx + Jy + D) / 2;
% e2 = (Jx + Jy - D) / 2; %第二个特征值
edge_magnitude = sqrt(e1);
edge_orientation = atan2(-Jxy, e1 - Jy);
% figure,
% subplot(121),imshow(edge_magnitude)
% subplot(122),imshow(edge_orientation)
sob=edge(edge_magnitude,'sobel',0.29);
% sob=bwareaopen(sob,100);
% figure,imshow(y),title('Sobel Edge Detection')
% 3*3 sobel
f=edge_magnitude;
sx=[-1 0 1;-2 0 2;-1 0 1]; % 卷积模板convolution mask
sy=[-1 -2 -1;0 0 0;1 2 1]; % 卷积模板convolution mask
for x=2:1:nI(1,1)-1
for y=2:1:nI(1,2)-1
mod=[f(x-1,y-1),2*f(x-1,y),f(x-1,y+1);
f(x,y-1),2*f(x,y),f(x,y+1);
f(x+1,y-1),2*f(x+1,y),f(x+1,y+1)];
mod=double(mod);
fsx=sx.*mod;
fsy=sy.*mod;
ftemp(x,y)=sqrt((sum(fsx(:)))^2+(sum(fsy(:)))^2);
end
end
fs=im2bw(ftemp); % fs=im2uint8(ftemp);
fs=bwareaopen(fs,500);
% figure,imshow(fs);title('Sobel Edge Detection')
subplot(131),imshow(edge_magnitude),title('edge magnitude')
subplot(132),imshow(sob),title('edge magnitude extraction')
subplot(133),imshow(fs);title('sobel Edge Detection')
operation result:
3.2 Prewitt operator edge detection
The Prewitt operator expands the size of the edge detection operator template from 2x2 to 3x3, calculates the difference operator, and combines the direction difference operator with the local average to reduce the influence of noise on image edge detection. It is defined as:
S x = [ f ( x − 1 , y + 1 ) + f ( x , y + 1 ) + f ( x + 1 , y + 1 ) ] − [ f ( x − 1 , y − 1 ) − f ( x , y − 1 ) − f ( x + 1 , y − 1 ) ] {S_x} = \left[ {f\left( {x{\rm{ - }}1{\rm{,} }y + 1} \right) + f\left( {x{\rm{,}}y + 1} \right) + f\left( {x + 1{\rm{,}}y + 1} \ right)} \right]{\rm{ - }}\left[ {f\left( {x{\rm{ - }}1{\rm{,}}y{\rm{ - }}1} \right ){\rm{ - }}f\left( {x{\rm{,}}y{\rm{ - }}1} \right){\rm{ - }}f\left( {x + 1{ \rm{,}}y{\rm{ - }}1} \right)} \right]Sx=[f(x−1,y+1)+f(x,y+1)+f(x+1,y+1)]−[f(x−1,y−1)−f(x,y−1)−f(x+1,y−1)]
S y = [ f ( x + 1 , y − 1 ) + f ( x + 1 , y ) + f ( x + 1 , y + 1 ) ] − [ f ( x − 1 , y − 1 ) − f ( x − 1 , y ) − f ( x − 1 , y + 1 ) ] {S_y} = \left[ {f\left( {x + 1{\rm{,}}y{\rm{ - }}1} \right) + f\left( {x + 1{\rm{,}}y} \right) + f\left( {x + 1{\rm{,}}y + 1} \right)} \right]{\rm{ - }}\left[ {f\left( {x{\rm{ - }}1{\rm{,}}y{\rm{ - }}1} \right){\rm{ - }}f\left( {x{\rm{ - }}1{\rm{,}}y} \right){\rm{ - }}f\left( {x{\rm{ - }}1{\rm{,}}y + 1} \right)} \right] Sy=[f(x+1,y−1)+f(x+1,y)+f(x+1,y+1)]−[f(x−1,y−1)−f(x−1,y)−f(x−1,y+1 ) ]
The convolution template of the Prewitt operator is:G ( i , j ) = ∣ P x ∣ + ∣ P y ∣ G(i,j)=|P_x|+|P_y|G(i,j)=∣Px∣+∣Py∣ , where:
The code of the Prewitt operator is as follows:
%% Prewitt
clc,clear,close all
I=imread('1.jpg'); % 读入图像
r=I(:,:,1);g=I(:,:,2);b=I(:,:,3);
nI=size(r);
im = single(I) / 255;
yfilter = fspecial('prewitt'); % prewitt
xfilter = yfilter';
rx = imfilter(im(:,:,1), xfilter);
gx = imfilter(im(:,:,2), xfilter);
bx = imfilter(im(:,:,3), xfilter);
ry = imfilter(im(:,:,1), yfilter);
gy = imfilter(im(:,:,2), yfilter);
by = imfilter(im(:,:,3), yfilter);
Jx = rx.^2 + gx.^2 + bx.^2;
Jy = ry.^2 + gy.^2 + by.^2;
Jxy = rx.*ry + gx.*gy + bx.*by;
D = sqrt(abs(Jx.^2 - 2*Jx.*Jy + Jy.^2 + 4*Jxy.^2)); % 2x2 matrix J'*J的第一个特征值
e1 = (Jx + Jy + D) / 2;
% e2 = (Jx + Jy - D) / 2; %第二个特征值
edge_magnitude = sqrt(e1);
edge_orientation = atan2(-Jxy, e1 - Jy);
% figure,
% subplot(121),imshow(edge_magnitude)
% subplot(122),imshow(edge_orientation)
pre=edge(edge_magnitude,'prewitt',0.19);
% figure,imshow(y),title('Prewitt Edge Detection')
% 3*3 prewitt
f=edge_magnitude;
sx=[-1 0 1;-1 0 1;-1 0 1]; % convolution mask
sy=[1 1 1;0 0 0;-1 -1 -1]; % convolution mask
for x=2:1:nI(1,1)-1
for y=2:1:nI(1,2)-1
mod=[f(x-1,y-1),f(x-1,y),f(x-1,y+1);
f(x,y-1),f(x,y),f(x,y+1);
f(x+1,y-1),f(x+1,y),f(x+1,y+1)];
mod=double(mod);
fsx=sx.*mod;
fsy=sy.*mod;
ftemp(x,y)=sqrt((sum(fsx(:)))^2+(sum(fsy(:)))^2);
end
end
fs=im2bw(ftemp); % fs=im2uint8(ftemp);
fs=bwareaopen(fs,1000);
% figure,imshow(fs);title('Prewitt Edge Detection');
subplot(131),imshow(edge_magnitude),title('edge magnitude')
subplot(132),imshow(pre),title('edge magnitude extraction')
subplot(133),imshow(fs);title('Prewitt Edge Detection');
operation result:
3.3 Canny operator edge detection
The traditional Canny algorithm uses the finite difference of the first-order partial derivative of the 2x2 neighborhood to calculate the gradient magnitude and gradient direction of the smoothed data array I(x,y). Defined as:
P x ( i , j ) = ( I ( i , j + 1 ) − I ( i , j ) + I ( i + 1 , j + 1 ) − I ( i + 1 , j ) ) / 2 {P_x}\left( {i,j} \right) = \left( {I\left( {i,j + 1} \right) - I\left( {i,j} \right) + I\left ( {i + 1,j + 1} \right) - I\left( {i + 1,j} \right)} \right)/2Px(i,j)=(I(i,j+1)−I(i,j)+I(i+1,j+1)−I(i+1,j))/2
P y ( i , j ) = ( I ( i , j ) − I ( i + 1 , j ) + I ( i , j + 1 ) − I ( i + 1 , j + 1 ) ) / 2 {P_y}\left( {i,j} \right) = \left( {I\left( {i,j} \right) - I\left( {i + 1,j} \right) + I\left( {i,j + 1} \right) - I\left( {i + 1,j + 1} \right)} \right)/2 Py(i,j)=(I(i,j)−I(i+1,j)+I(i,j+1)−I(i+1,j+1))The gradient amplitude and gradient direction of the /2
pixel are calculated by the coordinate conversion formula from rectangular coordinates to polar coordinates, and the second-order norm is used to calculate the gradient amplitude: M ( i , j ) =
P x ( i , j ) 2 + P y ( i , j ) 2 M\left( {i,j} \right) = \sqrt { {
P_x}{
{\left( {i,j} \right)}^2} + {P_y}{
{\left( {i,j} \right)}^2}}M(i,j)=Px(i,j)2+Py(i,j)2
梯度方向为：
θ [ i , j ] = arctan ( P y ( i , j ) / P x j ( i , j ) ) \theta \left[ {i,j} \right] = \arctan \left( {
{P_y}\left( {i,j} \right)/{P_{xj}}\left( {i,j} \right)} \right) i[i,j]=arctan(Py(i,j)/Pxj(i,j ) )
Canny operator is more effective for image edge detection. It is accurate to detect the noise on the edge of the sample and the welding position, but the quality texture of the weld surface is relatively regular, and the Canny detection texture is thinner, which increases the calculation workload. Greater impact. code show as below:
%% Canny
clc,clear,close all
im=imread('1.jpg'); % 载入图像
im=im2double(im);
r=im(:,:,1);g=im(:,:,2);b=im(:,:,3);
% 滤波器平滑系数
filter= [2 4 5 4 2;
4 9 12 9 4;
5 12 15 12 5;
4 9 12 9 4;
2 4 5 4 2];
filter=filter/115;
% N-dimensional convolution
smim= convn(im,filter);
% imshow(smim);title('Smoothened image');
% 计算梯度
gradXfilt=[-1 0 1; % 卷积模板convolution mask
-2 0 2;
-1 0 1];
gradYfilt=[1 2 1; % 卷积模板 convolution mask
0 0 0;
-1 -2 -1];
GradX= convn(smim,gradXfilt);
GradY= convn(smim,gradYfilt);
absgrad=abs(GradX)+abs(GradY);
% computing angle of gradients
[a,b]=size(GradX);
theta=zeros([a b]);
for i=1:a
for j=1:b
if(GradX(i,j)==0)
theta(i,j)=atan(GradY(i,j)/0.000000000001);
else
theta(i,j)=atan(GradY(i,j)/GradX(i,j));
end
end
end
theta=theta*(180/3.14);
for i=1:a
for j=1:b
if(theta(i,j)<0)
theta(i,j)= theta(i,j)-90;
theta(i,j)=abs(theta(i,j));
end
end
end
for i=1:a
for j=1:b
if ((0<theta(i,j))&&(theta(i,j)<22.5))||((157.5<theta(i,j))&&(theta(i,j)<181))
theta(i,j)=0;
elseif (22.5<theta(i,j))&&(theta(i,j)<67.5)
theta(i,j)=45;
elseif (67.5<theta(i,j))&&(theta(i,j)<112.5)
theta(i,j)=90;
elseif (112.5<theta(i,j))&&(theta(i,j)<157.5)
theta(i,j)=135;
end
end
end
%non-maximum suppression
nmx = padarray(absgrad, [1 1]);
[a,b]=size(theta);
for i=2:a-2
for j=2:b-2
if (theta(i,j)==135)
if ((nmx(i-1,j+1)>nmx(i,j))||(nmx(i+1,j-1)>nmx(i,j)))
nmx(i,j)=0;
end
elseif (theta(i,j)==45)
if ((nmx(i+1,j+1)>nmx(i,j))||(nmx(i-1,j-1)>nmx(i,j)))
nmx(i,j)=0;
end
elseif (theta(i,j)==90)
if ((nmx(i,j+1)>nmx(i,j))||(nmx(i,j-1)>nmx(i,j)))
nmx(i,j)=0;
end
elseif (theta(i,j)==0)
if ((nmx(i+1,j)>nmx(i,j))||(nmx(i-1,j)>nmx(i,j)))
nmx(i,j)=0;
end
end
end
end
nmx1=im2uint8(nmx); % 图像数据类型转换
tl=85; % 阈值下限lower threshold
th=100; % 阈值上限upper threshold
% grouping edges based on threshold
[a,b]=size(nmx1);
gedge=zeros([a,b]);
for i=1:a
for j=1:b
if(nmx1(i,j)>th)
gedge(i,j)=nmx1(i,j);
elseif (tl<nmx1(i,j))&&(nmx1(i,j)<th)
gedge(i,j)=nmx1(i,j);
end
end
end
[a,b]= size(gedge);
finaledge=zeros([a b]);
for i=1:a
for j=1:b
if (gedge(i,j)>th)
finaledge(i,j)=gedge(i,j);
for i2=(i-1):(i+1)
for j2= (j-1):(j+1)
if (gedge(i2,j2)>tl)&&(gedge(i2,j2)<th)
finaledge(i2,j2)=gedge(i,j);
end
end
end
end
end
end
%clearing the border
finaledge= im2uint8(finaledge(10:end-10,10:end-10));
subplot(131);imshow(absgrad);title('image gradients');
subplot(132);imshow(nmx);title('NonMaximum Suppression');
subplot(133);imshow(finaledge(:,1:452-10));title('canny edge detection');
operation result:
3.4 Morphological Processing Edge Detection
The process of morphological processing edge detection is as follows:
the running results are as follows:
4. Results analysis
It can be seen from the above figure that the Sobel operator detects the edge of the image, and the detection result is more detailed, and the background contour interference is too large, so that the target weld cannot be single-extracted. ; The Prewitt operator has a good extraction effect on the image edge, but the weld seam is greatly affected by the background, and the extraction effect is poor, and there is no closed area at the edge of the extraction; the Canny operator is based on the non-maximum value suppression of the double threshold, and the obtained Compared with the Sobel and Prewit operators, the edge image detects finer textures, but since there is no closed area, it is still difficult to remove the background edge and extract the weld outline alone; The filtering is good, and the obtained binary image is close to the actual target, which ensures that the feature information of the weld is not lost, so that the feature of the weld can be extracted well, with a large signal-to-noise ratio and high precision.
Ok, the above is the whole content of this article, the complete code can refer to: https://download.csdn.net/download/didi_ya/87743384 ; if it is helpful to you, remember to like it~