Vehicle detection based on YOLOv4 MATLAB implementation

If you need the source program, you can follow me and comment and I will send it to you via email~

Table of contents

Summary

Research Background

Algorithm design and implementation process

Construction of vehicle target data set

Target detection based on YOLOv4

Improve the YOLOv4 model

Experimental results and analysis

Conclusion and Outlook

Code

---

Summary

       For vehicle detection, this paper proposes a method based on YOLOv4Vehicle detection algorithm. A multi-day, multi-time period, and multi-scenario vehicle target data set was produced. The vehicle data set was manually annotated and divided into a training set and a test set for model use. ThroughDarkNet53 The network framework was used for training and it was found that the experimental results were good and could meet the needs of practical applications.

Research Background

       With the development of the economy, the problem of traffic congestion has become prominent, which has seriously affected the improvement of people's living standards. Building an intelligent traffic monitoring system is of great significance to reducing traffic congestion and improving transportation efficiency. Accurate and real-time detection of vehicle targets is the core of intelligent transportation systems. In the research on existing vehicle detection algorithms, detection algorithms based on deep learning have successfully attracted the attention of scholars, especially for the detection of multiple vehicles in complex scenes. More challenging.

Algorithm design and implementation process

Figure 1 Algorithm design block diagram

       Deep learning technology mainly performs convolution processing on the input image, extracts features and determines the target location. It has good results in target recognition. The main representative algorithms of target recognition algorithms based on deep learning include R-CNN, SSD, YOLO, etc. This article uses the YOLOv4 algorithm, which is a typical representative of one-stage detection. It has good real-time performance and fast detection speed. Based on the above advantages, we finally chose it.

Construction of vehicle target data set

       We consider that the vehicles on the road and the environment they are in are relatively complex, which will affect the accuracy of vehicle detection, so

       We believe that it is very important to build a suitable real-time monitoring data set. COCO's vehicle data set is collected online, and the vehicle data set is found. After storage, image combination and bounding box creation are performed, and the box label data is stored together.

Figure 2 Vehicle data set

Target detection based on YOLOv4

       The YOLOv4 algorithm is optimized based on the target detection algorithm of previous versions of the YOLO series and is relatively popular in the field of target recognition. It mainly uses CSPDarknet53 as the backbone feature extraction network, in which there are five large residual blocks, each large The number of small residual units contained in the residual block is 1, 2, 8, 8, 4. The purpose of the SPP network used in YOLOv4 is to enhance the receptive field of the network. PANet was added after the upsampling process. In the downsampling operation, YOLOv3 is used as the Head, and finally the entire YOLOv4 algorithm is implemented.

Figure 3 YOLOv4 network structure diagram

Set the training hyperparameters as follows, numEpochs = 90; miniBatchSize = 4; learningRate = 0.001; warmupPeriod = 1000; l2Regularization = 0.001; penaltyThreshold = 0.5;

The stochastic gradient descent method (Stochastic Gradient Descent, SGD) is used. The training process is fine-tuned under YOLOv4 weights and iterated 100 times to complete the training process.

Then run the detector on the images in the test set and collect the results. Use the average accuracy metric to evaluate the object detector. The average accuracy after training on the test set is 86%, and save the trained model and parameters after training.

Finally, create a detection file, download the pre-downloaded network, obtain the class name of the COCO data set and the anchors used for training and pre-training models, and then perform detection and visualization. The results are as follows.

Figure 4 Test result 1

       After we got the results, we found that the detection speed was slow during the detection process. The detection results did not appear until about 30 seconds after running the program, and the small vehicles behind them could not be selected and recognized during the detection process, and could not be recognized. Vehicle video streaming. We improved the model based on the problems we found.

Improve the YOLOv4 model

       After we passed the first round of training, we found that our model has some flaws in the accuracy of small target recognition and the speed of detection. So we optimize and improve the YOLOv4 model.

      First of all, in terms of detection speed, we want to make the YOLOv4 model lightweight to solve this problem. CSPDarknet53 is the YOLOv4 backbone feature extraction network. The calculation results of the two convolutional layers in this lightweight structure are directly channel fused, and then share a BN layer and activation function. This can reduce the calculation amount of the network and avoid the two convolutional modules. Activation values ​​and batch normalization are calculated separately. While making the model lightweight, it will not affect the final experimental results.

       In the process of small target recognition, we consider that it may be caused by more single vehicles or single environments in the data set, so we use the data enhancement method to expand the data. Data enhancement refers to sending the pictures to the network. In the pre-processing stage, by making some minor changes to the input image, such as random erasing, adjusting saturation, etc., the neural network thinks that this is a brand new image to increase the amount of data. This can not only increase the amount of target detection data, but also enrich the background of the detected targets in the data set and reduce the probability that the background is misjudged as the detection target.​ 

Figure 5 Data enhancement process diagram

Experimental results and analysis

       After our improvements, we got an accuracy rate of 91% on the test set after running the program. The detection results can be obtained in about 20 seconds, and small vehicles can also be better recognized and detected. And we separate the video into each frame. Similarly The identification of video streams is also implemented. By training and verifying the model, it was found that the improved model was better than the original model in terms of accuracy and detection speed.

Figure 6 Test result 2

Conclusion and Outlook

       We have completed the recognition of vehicles in traffic video pictures, analyzed the vehicle recognition principles, and improved the vehicle detection algorithm. The performance of the improved YOLOv4 model has been greatly improved, and the vehicle detection accuracy is high. Through experimental testing, it was found that the vehicle detection model can meet the requirements of real-time detection, but there is still room for improvement and optimization of the vehicle detection model. The following aspects can also be optimized:

       Most of the pictures used for vehicle recognition are taken during the day, and the statistical effect on vehicles during the day is better. The follow-up research direction will conduct comparative analysis on snowy days, rainy days, night and other environments.

       During the recognition process, we found that many small vehicles are still undetectable. We need to continue to optimize the model and train models with better accuracy to avoid this situation.

Code

clc;
clear;
close all;
warning off;
addpath(genpath(pwd));

%% 下载预训练网络
% 设置上述模型名称以下载该预训练模型。
modelName = 'YOLOv4-coco';
model = helper.downloadPretrainedYOLOv4(modelName);
net = model.net;

%% 加载数据
%解压缩车辆图像并加载车辆地面实况数据。
unzip vehicleDatasetImages.zip
data = load('vehicleDatasetGroundTruth.mat');
vehicleDataset = data.vehicleDataset;

% 将完整路径添加到本地车辆数据文件夹。
vehicleDataset.imageFilename = fullfile(pwd, vehicleDataset.imageFilename);


rng('default')
shuffledIndices = randperm(height(vehicleDataset));
idx = floor(0.6 * length(shuffledIndices));
trainingDataTbl = vehicleDataset(shuffledIndices(1:idx), :);
testDataTbl = vehicleDataset(shuffledIndices(idx+1:end), :);

% 创建用于加载图像的图像数据存储.
imdsTrain = imageDatastore(trainingDataTbl.imageFilename);
imdsTest = imageDatastore(testDataTbl.imageFilename);
 
% 为地面真相边界框创建数据存储。
bldsTrain = boxLabelDatastore(trainingDataTbl(:, 2:end));
bldsTest = boxLabelDatastore(testDataTbl(:, 2:end));

% 组合图像和框标签数据存储。
trainingData = combine(imdsTrain, bldsTrain);
testData = combine(imdsTest, bldsTest);

helper.validateInputData(trainingData);
helper.validateInputData(testData);

%% 数据扩充
augmentedTrainingData = transform(trainingData, @helper.augmentData);
 
% 增强后的图片
augmentedData = cell(4,1);
for k = 1:4
    data = read(augmentedTrainingData);
    augmentedData{k} = insertShape(data{1,1}, 'Rectangle', data{1,2});
    reset(augmentedTrainingData);
end
figure
montage(augmentedData, 'BorderSize', 10)

%% 预处理训练数据
% 指定网络输入大小。 
networkInputSize = net.Layers(1).InputSize;
 

preprocessedTrainingData = transform(augmentedTrainingData, @(data)helper.preprocessData(data, networkInputSize));
 
% 读取预处理的训练数据。
data = read(preprocessedTrainingData);

% 显示带有边界框的图像。
I = data{1,1};
bbox = data{1,2};
annotatedImage = insertShape(I, 'Rectangle', bbox);
annotatedImage = imresize(annotatedImage,2);
figure
imshow(annotatedImage)

% 重置数据存储。
reset(preprocessedTrainingData);

%% 修改预训练YOLO v4网络

rng(0)
trainingDataForEstimation = transform(trainingData, @(data)helper.preprocessData(data, networkInputSize));
numAnchors = 9;
[anchorBoxes, meanIoU] = estimateAnchorBoxes(trainingDataForEstimation, numAnchors);

% 指定培训中要使用的className。
classNames = {'vehicle'};


[lgraph, networkOutputs, anchorBoxes, anchorBoxMasks] = configureYOLOv4(net, classNames, anchorBoxes, modelName);

%% 指定超参数
numEpochs = 200;
miniBatchSize = 4;
learningRate = 0.01;
warmupPeriod = 1000;
l2Regularization = 0.001;
penaltyThreshold = 0.5;
velocity = [];

%% 训练模型
if canUseParallelPool
   dispatchInBackground = true;
else
   dispatchInBackground = false;
end

mbqTrain = minibatchqueue(preprocessedTrainingData, 2,...
        "MiniBatchSize", miniBatchSize,...
        "MiniBatchFcn", @(images, boxes, labels) helper.createBatchData(images, boxes, labels, classNames), ...
        "MiniBatchFormat", ["SSCB", ""],...
        "DispatchInBackground", dispatchInBackground,...
        "OutputCast", ["", "double"]);



% 将层图转换为dlnetwork。
net = dlnetwork(lgraph);

% 学习率和batch_size子图。
fig = figure;
[lossPlotter, learningRatePlotter] = helper.configureTrainingProgressPlotter(fig);

iteration = 0;
% 自定义训练循环。
for epoch = 1:numEpochs
      
    reset(mbqTrain);
    shuffle(mbqTrain);
    
    while(hasdata(mbqTrain))
        iteration = iteration + 1;
       
        [XTrain, YTrain] = next(mbqTrain);
        
        % 使用dlfeval和modelGradients函数评估模型梯度和损失。
        [gradients, state, lossInfo] = dlfeval(@modelGradients, net, XTrain, YTrain, anchorBoxes, anchorBoxMasks, penaltyThreshold, networkOutputs);

        % 应用L2正则化。
        gradients = dlupdate(@(g,w) g + l2Regularization*w, gradients, net.Learnables);

        % 确定当前学习率
        currentLR = helper.piecewiseLearningRateWithWarmup(iteration, epoch, learningRate, warmupPeriod, numEpochs);
        
        % 使用SGDM优化器更新网络可学习参数。
        [net, velocity] = sgdmupdate(net, gradients, velocity, currentLR);

        % 更新dlnetwork的状态参数。
        net.State = state;
        
        % 显示进度。
        if mod(iteration,10)==1
            helper.displayLossInfo(epoch, iteration, currentLR, lossInfo);
        end
            
        % 更新训练图。
        helper.updatePlots(lossPlotter, learningRatePlotter, iteration, currentLR, lossInfo.totalLoss);
    end
end

% 保存训练模型。
anchors.anchorBoxes = anchorBoxes;
anchors.anchorBoxMasks = anchorBoxMasks;

save('yolov4_trained', 'net', 'anchors');

%% 评估模型
confidenceThreshold = 0.5;
overlapThreshold = 0.5;

%创建一个表以保存返回的边界框、分数和标签检测器。
numImages = size(testDataTbl, 1);
results = table('Size', [0 3], ...
    'VariableTypes', {'cell','cell','cell'}, ...
    'VariableNames', {'Boxes','Scores','Labels'});

% 对测试集中的图像运行检测器并收集结果。
reset(testData)
while hasdata(testData)
    % 读取数据存储并获取图像。
    data = read(testData);
    image = data{1};
    
    % 运行预测器
    executionEnvironment = 'auto';
    [bboxes, scores, labels] = detectYOLOv4(net, image, anchors, classNames, executionEnvironment);
    
    % 收集结果。
    tbl = table({bboxes}, {scores}, {labels}, 'VariableNames', {'Boxes','Scores','Labels'});
    results = [results; tbl];
end

% 使用平均精度度量评估对象检测器。
[ap, recall, precision] = evaluateDetectionPrecision(results, testData);

%精确召回（PR）曲线显示了检测器在变化时的精度召回水平。理想情况下，所有召回级别的精度均为1。

% 绘制精度召回曲线。
figure
plot(recall, precision)
xlabel('Recall')
ylabel('Precision')
grid on
title(sprintf('Average Precision = %.2f', ap))

%% 使用经过训练的YOLO v4检测对象
reset(testData)
data = read(testData);

% 选取图片
I = data{1};

% 运行预测
executionEnvironment = 'auto';
[bboxes, scores, labels] = detectYOLOv4(net, I, anchors, classNames, executionEnvironment);

% 预测图片
if ~isempty(scores)
    I = insertObjectAnnotation(I, 'rectangle', bboxes, scores);
end
figure
imshow(I)