Surface defect detection is a very important research content in the field of machine vision, also known as AOI (Automated optical inspection) or AsI (Automated surface inspection). Defective technology. At present, surface defect equipment based on machine vision has widely replaced manual visual inspection in various industrial fields, including 3C, automobile, home appliance, machinery manufacturing, semiconductor and electronics, chemical industry, medicine, aerospace, light industry and other industries. Traditional surface defect detection methods based on machine vision often use conventional image processing algorithms or artificially designed features plus classifiers.
1. Algorithm Description
In recent years, with the successful application of deep learning models represented by convolutional neural network (fConv01utional neural network, CNN) in many computer vision (Computer vjsion, CV) fields, such as face recognition, pedestrian recognition, scene text detection, target recognition. Many deep learning-based defect detection methods are also widely used in various industrial scenarios, such as tracking and autonomous driving, and even some domestic and foreign companies have developed a variety of deep learning-based commercial industrial surface defect detection software, as shown in Table 1. . The global market size of traditional industrial vision and its components will reach US$19.2 billion in 2025 [", of which China accounts for about 30% and maintains an average annual growth rate of 14%. This field is gradually being replaced by a new generation based on depth. At the same time, my country proposed in the white paper "Made in China 2025" to "promote the use of advanced molding and processing methods, online testing devices, intelligent production and logistics systems and testing equipment, etc., so that the performance stability of key physical products can be improved. , quality reliability, environmental adaptability, service life and other indicators have reached the international advanced level of similar products."
In recent years, surface defect detection technology based on deep learning has been widely used in various industrial scenarios. This paper sorts out the surface defect detection methods based on deep learning in recent years, and divides them into three categories: fully supervised learning model methods, unsupervised learning model methods and other methods according to the different data labels. Subdivided classification and comparative analysis, summed up the advantages and disadvantages and application scenarios of each method.
In the process of surface defect detection of metal mechanical parts, cascade classifiers need to be introduced to classify the output features in all connection layers, so that more accurate classification results can be obtained. In order to fully extract the features of the fully connected layer, the gradient histogram of the convolutional neural network and the local binary pattern are used to extract the output features, and at the same time, multiple different cascaded classifiers are trained in turn, and the obtained classification results are fused for decision-making. The result of decision fusion realizes part surface defect detection. The directional gradient histogram has good invariant characteristics under the features of subtle deformation, but the ability of the directional gradient histogram to describe local features is flawed, and it is very sensitive to noise. The local binary mode has better local expression ability, so the directional gradient histogram and the local binary mode have better complementarity. When the features of the original metal mechanical parts image are extracted through the convolutional neural network, the extracted features are counted and put into the fully connected layer at the same time. The complementary features of HOG and LBP are fully utilized to further extract the output features of HOG. The eigenvalues extracted by the above methods have relatively strong robust characteristics.
Second, the code implementation
function varargout = main(varargin)
% MAIN MATLAB code for main.fig
% MAIN, by itself, creates a new MAIN or raises the existing
% singleton*.
%
% H = MAIN returns the handle to a new MAIN or the handle to
% the existing singleton*.
%
% MAIN('CALLBACK',hObject,eventData,handles,...) calls the local
% function named CALLBACK in MAIN.M with the given input arguments.
%
% MAIN('Property','Value',...) creates a new MAIN or raises the
% existing singleton*. Starting from the left, property value pairs are
% applied to the GUI before main_OpeningFcn gets called. An
% unrecognized property name or invalid value makes property application
% stop. All inputs are passed to main_OpeningFcn via varargin.
%
% *See GUI Options on GUIDE's Tools menu. Choose "GUI allows only one
% instance to run (singleton)".
%
% See also: GUIDE, GUIDATA, GUIHANDLES
% Edit the above text to modify the response to help main
% Last Modified by GUIDE v2.5 18-Apr-2022 10:33:59
% Begin initialization code - DO NOT EDIT
gui_Singleton = 1;
gui_State = struct('gui_Name', mfilename, ...
'gui_Singleton', gui_Singleton, ...
'gui_OpeningFcn', @main_OpeningFcn, ...
'gui_OutputFcn', @main_OutputFcn, ...
'gui_LayoutFcn', [] , ...
'gui_Callback', []);
if nargin && ischar(varargin{1})
gui_State.gui_Callback = str2func(varargin{1});
end
if nargout
[varargout{1:nargout}] = gui_mainfcn(gui_State, varargin{:});
else
gui_mainfcn(gui_State, varargin{:});
end
% End initialization code - DO NOT EDIT
% --- Executes just before main is made visible.
function main_OpeningFcn(hObject, eventdata, handles, varargin)
% This function has no output args, see OutputFcn.
% hObject handle to figure
% eventdata reserved - to be defined in a future version of MATLAB
% handles structure with handles and user data (see GUIDATA)
% varargin command line arguments to main (see VARARGIN)
% Choose default command line output for main
handles.output = hObject;
set(handles.radiobutton1,'value',0);
set(handles.radiobutton2,'value',0);
axes(handles.axes1); %指定需要清空的坐标轴
cla reset;
box on;
set(handles.axes1,'xtick',[]);
set(handles.axes1,'ytick',[]);
axes(handles.axes2); %指定需要清空的坐标轴
cla reset;
box on; %在坐标轴四周加上边框
set(handles.axes2,'xtick',[]);
set(handles.axes2,'ytick',[]);
axes(handles.axes3); %指定需要清空的坐标轴
cla reset;
box on; %在坐标轴四周加上边框
set(handles.axes3,'xtick',[]);
set(handles.axes3,'ytick',[]);
set(handles.edit1,'string','');
set(handles.edit2,'string','');
set(handles.edit3,'string','');
set(handles.edit4,'string','');
% Update handles structure
guidata(hObject, handles);
% UIWAIT makes main wait for user response (see UIRESUME)
% uiwait(handles.figure1);
% --- Outputs from this function are returned to the command line.
function varargout = main_OutputFcn(hObject, eventdata, handles)
% varargout cell array for returning output args (see VARARGOUT);
% hObject handle to figure
% eventdata reserved - to be defined in a future version of MATLAB
% handles structure with handles and user data (see GUIDATA)
% Get default command line output from handles structure
varargout{1} = handles.output;
% --- Executes on button press in pushbutton1.
function pushbutton1_Callback(hObject, eventdata, handles)
% hObject handle to pushbutton1 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB
% handles structure with handles and user data (see GUIDATA)
clc;
clear;
output=VOCxml_to_matlab_main();
save dataSet output;
% --- Executes on button press in pushbutton2.
function pushbutton2_Callback(hObject, eventdata, handles)
% hObject handle to pushbutton2 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB
% handles structure with handles and user data (see GUIDATA)
global inputSize;
data=load('dataSet');
data=data.output;
% 数据存储在一个包含三列的表中，其中第一列包含图像文件路径，第二列包含车辆边界框，
% 第三列对应的类别。
% 将数据集分成训练集、验证集和测试集。选择 60% 的数据用于训练，10% 用于验证，
% 其余用于测试经过训练的检测器
shuffledIndices = randperm(height(data));
idx = floor(0.6 * height(data));
trainingIdx = 1:idx;
trainingDataTbl = data(shuffledIndices(trainingIdx),:);
validationIdx = idx+1 : idx + 1 + floor(0.1 * length(shuffledIndices) );
validationDataTbl = data(shuffledIndices(validationIdx),:);
testIdx = validationIdx(end)+1 : length(shuffledIndices);
testDataTbl = data(shuffledIndices(testIdx),:);
% 使用 imageDatastore 和 boxLabelDatastore 创建数据存储，
% 以便在训练和评估期间加载图像和标签数据
imdsTrain = imageDatastore(trainingDataTbl{:,'imageFilename'});
bldsTrain = boxLabelDatastore(trainingDataTbl(:,2:end));
imdsValidation = imageDatastore(validationDataTbl{:,'imageFilename'});
bldsValidation = boxLabelDatastore(validationDataTbl(:,2:end));
imdsTest = imageDatastore(testDataTbl{:,'imageFilename'});
bldsTest = boxLabelDatastore(testDataTbl(:,2:end));
%组合图像和边界框标签数据存储
trainingData = combine(imdsTrain,bldsTrain);
validationData = combine(imdsValidation,bldsValidation);
testData = combine(imdsTest,bldsTest);
%显示其中一个训练图像和边界框标签
data = read(trainingData);
I = data{1};
bbox = data{2};
annotatedImage = insertShape(I,'Rectangle',bbox);
annotatedImage = imresize(annotatedImage,2);
figure
imshow(annotatedImage)
%是否需要训练
doTraining=true;
end
references
[1] Huang Fengrong, Li Yang. Part surface defect detection algorithm based on Faster RCNN.
[2] Lu Yingying, Sun Yu. Surface defect detection system of electroplated parts based on machine vision.
For details, please click : 1341703358 . In this paper, three key issues in surface defect detection are discussed, and a common dataset for industrial surface defects is introduced. Finally, the future development trend of surface defect detection is prospected.