Series Article Directory

Article directory

Series Article Directory
foreword
1. NoMachine, FileZilla and servers
2. Install and configure darknet
3. Dataset and model preparation
4. Training and testing
Five, cfg file interpretation

foreword

darkent is a lightweight deep learning training framework written in c and cuda and supports GPU acceleration. You can understand that darknet, like tensorflow, pytorch, caffe, and mxnet, is the bottom layer for running models. Like resnet and yolo, it is a model structure and a training network.

darknet is a deep learning framework written by the author of YOLO (see YOLO original text 2.2), and later in YOLO9000, a 19-layer convolutional network based on ResNet magic modification, called Darknet-19, was mentioned in YOLOv3 A deeper Darknet-53, both of which are backbone networks for feature extraction. Things like yolov2, v3, v4, and v7 can all be implemented on the Darknet framework (yolov5 uses pytorch). Its main feature is easy installation, no dependencies (OpenCV can be used), very good portability, and supports both CPU and GPU computing methods.

The use of the underlying C language is conducive to the improvement of operating efficiency, and the dependence on some algorithm libraries is conducive to reducing the volume. It has good portability and independence and can be used for embedded and other low-computing and low-cost boards (not including training).

Compared with the big frameworks such as TensorFlow and pytorch, darknet is not so powerful, but darknet also has its own advantages:
1. Easy to install: select the additional items you need (cuda, cudnn, opencv, etc.) in the makefile and directly make Yes, the installation can be completed in a few minutes;
2. No dependencies: the entire framework is written in C language and does not depend on any library. Even the author of opencv has written a function that can replace it; 3. The structure is clear and the source
code Easy to view and modify: the basic files of the framework are in the src folder, and some detection and classification functions defined are in the example folder, you can directly view and modify the source code as needed; 4. Friendly python interface: although
darknet It is written in c language, but it also provides a python interface. Through the python function, you can use python to directly call the trained model in .weight format; 5. Easy to transplant: the framework is very
simple to deploy locally on the machine, and can According to the machine situation, using cpu and gpu, especially for local deployment of detection and recognition tasks, darknet will be extremely convenient.

1. NoMachine, FileZilla and servers

Some that are not the focus of this article but will be used:

NoMachine installation and use
NoMachine is used to connect to the server remotely, provided that your server allows external network connections.

FileZilla installation and use
FileZilla is used for local and server connections for file transfer, provided that your server allows external network connections.

The server uses multiple users to avoid affecting the main user.

2. Install and configure darknet

1. Download

github official website AB version

(Design with a sense of dark net and black technology)
darknet magic circle
darknet official website

2. Compile

First of all, you must have your own deep learning environment. You can refer to my previous blog to configure the specific environment installation configuration , and then modify the Makefile value according to your own configuration environment.

Problem 1: NVCC problem

nvcc -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=[sm_50,compute_50] -gencode arch=compute_52,code=[sm_52,compute_52] -Iinclude/ -Isrc/ -DOPENCV pkg-config --cflags opencv -DGPU -I/usr/local/cuda/include/ -DCUDNN --compiler-options “-Wall -Wno-unused-result -Wno-unknown-pragmas -Wfatal-errors -fPIC -Ofast -DOPENCV -DGPU -DCUDNN” -c ./src/convolutional_kernels.cu -o obj/convolutional_kernels.o
/bin/sh: 1: nvcc: not found
Makefile:89: recipe for target ‘obj/convolutional_kernels.o’ failed
make: *** [obj/convolutional_kernels.o] Error 127

Solution:

Enter the darknet directory, edit the Makefile, and modify the NVCC path.
insert image description here

Execute the command under the darknet folder

make

compile successfully

Question 2: yolo_console_dll.cpp:(.text.startup+0x2ec) problem

When make compiles darknet, an error is reported

在函数‘main’中： yolo_console_dll.cpp:(.text.startup+0x2ec)：对‘Detector::Detector(std::__cxx11::basic_string, std::allocator >, std::__cxx11::basic_string, std::allocator >, int, int)’未定义的引用 collect2: error: ld returned 1 exit status Makefile:173: recipe for target 'uselib' failed make: *** [uselib] Error 1

Solution:

Refer to #7654
into the Makefile

note

#$(CPP) -std=c++11 $(COMMON) $(CFLAGS) -o $@ src/yolo_console_dll.cpp $(LDFLAGS) -L ./ -l:$(LIBNAMESO)

change to

$(CPP) -std=c++11 $(COMMON) $(CFLAGS) -o $@ src/yolo_console_dll.cpp $(LDFLAGS) -L ./ $(LIBNAMESO)

3. Dataset and model preparation

1. Data set download

Source: Baseline System of the Smart Blind Guide Group of the 15th China Computer Design Competition

structure:

WisdomGuide______annotations______train_list.txt
		     |			      |___val_list.txt
		     |			      |___instance_train.json
		     |			      |___instance_val.json
		     |_______train________JPEGImages___图片文件
 		     |                |___labels___标签文件
		     |________val________JPEGImages___图片文件
		                      |___labels___标签文件

Modify the distribution of the data set, create new train, val, JPEGImages and labels folders, where train stores the training set, val stores the test set, JPEGImages stores pictures, labels stores label files, train_list.txt is the path of all training pictures, val_list.txt are all test image paths. The establishment and distribution of these files are regular and have a reason. If you don't want to modify the code, use this data set format.

Note that for the picture folder JPEGImages and the label folder labels, if you want to separate the pictures and labels into different folders, then the name of the folder must be JPEGImages and labels, if you want to mix pictures and labels in the same folder Then don't use these two names.

The train_list.txt, val_list.txt, label txt files are generated by the script below.

2. json to txt script and path aggregation script

json_txt.py:
Change to your own path


import os
import json
from tqdm import tqdm
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('--json_path', default='/home/nh666/llw/ORB_FAR/WisdomGuide/annotations/instance_val.json', type=str, help="input: coco format(json)")
parser.add_argument('--save_path', default='/home/nh666/llw/ORB_FAR/WisdomGuide/annotations/val_txt', type=str, help="specify where to save the output dir of labels")
arg = parser.parse_args()


def convert(size, box):
    dw = 1. / (size[0])
    dh = 1. / (size[1])
    x = box[0] + box[2] / 2.0
    y = box[1] + box[3] / 2.0
    w = box[2]
    h = box[3]

    x = x * dw
    w = w * dw
    y = y * dh
    h = h * dh
    return (x, y, w, h)


if __name__ == '__main__':
    json_file = arg.json_path  # COCO Object Instance 类型的标注
    ana_txt_save_path = arg.save_path  # 保存的路径

    data = json.load(open(json_file, 'r'))
    if not os.path.exists(ana_txt_save_path):
        os.makedirs(ana_txt_save_path)

    id_map = {
    
    }  # coco数据集的id不连续！重新映射一下再输出！
    for i, category in enumerate(data['categories']):
        id_map[category['id']] = i

    # 通过事先建表来降低时间复杂度
    max_id = 0
    for img in data['images']:
        max_id = max(max_id, img['id'])
    # 注意这里不能写作 [[]]*(max_id+1)，否则列表内的空列表共享地址
    img_ann_dict = [[] for i in range(max_id + 1)]
    for i, ann in enumerate(data['annotations']):
        img_ann_dict[ann['image_id']].append(i)

    for img in tqdm(data['images']):
        filename = img["file_name"]
        img_width = img["width"]
        img_height = img["height"]
        img_id = img["id"]
        head, tail = os.path.splitext(filename)
        ana_txt_name = head + ".txt"  # 对应的txt名字，与jpg一致
        f_txt = open(os.path.join(ana_txt_save_path, ana_txt_name), 'w')
        '''for ann in data['annotations']:
            if ann['image_id'] == img_id:
                box = convert((img_width, img_height), ann["bbox"])
                f_txt.write("%s %s %s %s %s\n" % (id_map[ann["category_id"]], box[0], box[1], box[2], box[3]))'''
        # 这里可以直接查表而无需重复遍历
        for ann_id in img_ann_dict[img_id]:
            ann = data['annotations'][ann_id]
            box = convert((img_width, img_height), ann["bbox"])
            f_txt.write("%s %s %s %s %s\n" % (id_map[ann["category_id"]], box[0], box[1], box[2], box[3]))
        f_txt.close()

path_synthesis.py:
Change to your own path

# -*- coding: utf-8 -*-
import time
import os
import shutil

# 获取所有文件路径集合

def readFilename(path, allfile):
    filelist = os.listdir(path)

    for filename in filelist:
        filepath = os.path.join(path, filename)
        if os.path.isdir(filepath):
            readFilename(filepath, allfile)
        else:
            allfile.append(filepath)
    return allfile


if __name__ == '__main__':
    # 文件夹路径
    path1 = "/home/nh666/llw/ORB_FAR/WisdomGuide/val"
    allfile1 = []
    allfile1 = readFilename(path1, allfile1)
    allname1 = []
    # 文件路径
    txtpath = "/home/nh666/llw/ORB_FAR/WisdomGuide/annotations/val_list.txt"
    for name in allfile1:
        print(name)
        file_cls = name.split("/")[-1].split(".")[-1]
        # 后缀名
        if file_cls == 'png':
            print(name.split("/")[-1])
            with open(txtpath, 'a+') as fp:
                fp.write("".join(name) + "\n")

3. File modification

1. The .names file

Copy a darknet/data/voc.names file and rename it to voc_blind.names, which stores the training category, but it will not really mark why or what during training, but mark it as the serial number 01234, so in the file The sorting of category names is very important, to correspond to your tag txt file.

insert image description here

2. .data file

Copy a darknet/cfg/voc.data file and rename it to voc_blind.data, which stores the number of classes, train/valid dataset path, names category name path, and backup model path, which need to be modified according to your own location.
insert image description here

3. .cfg file

A few days ago, yolov7 was released, and it was found that darknet has also been implemented. I originally planned to use v7, but found that the training could not be performed during training, and an error was reported.

 cuDNN status Error in: file: ./src/convolutional_kernels.cu : () : line: 555 : build time: Oct 26 2022 - 16:19:38 
 cuDNN Error: CUDNN_STATUS_BAD_PARAM
Darknet error location: ./src/dark_cuda.c, cudnn_check_error, line #204
cuDNN Error: CUDNN_STATUS_BAD_PARAM: Resource temporarily unavailable

Solution ideas:
There are three main ideas

Change the version of CUDA and CUDNN, which version is not clear
Do not use the -map command during training, do not calculate mAP
Change and adjust the value of batch and subdivisions, it is not clear how much it is

I chose to change the batch and subdivisions to 64 in the cfg to solve this problem. This change seems to be related to your training hardware, such as single card, multi-card, cpu, gpu, etc.

batch=64
subdivisions=64  # changed from 16 to 64
width=416
height=416

As for CUDA and CUDNN, I checked the discussion area in github . It seems that I need to change the cuDNN version, but I am training on the server and cannot change it casually, so I use yolov4 instead.

4. yolov4-tiny.cfg

Copy a darknet/cfg/yolov4-tiny.cfg file and rename it to yolov4_blind.cfg, change the settings:
subdivisions is a multiple of 8, if the GPU memory is large, subdivisions can be filled with 8, and the memory hours can be filled with 32.
max_batches is the maximum number of training times, which is not directly related to the number of samples. It is recommended to be the number of samples × 2000 to avoid fitting problems.
The steps are changed to 80% and 90% of max_batches.

subdivisions=8
max_batches = 10000
steps=8000,9000

Ctrl+F searches for the "yolo" keyword, there are a few to change, and continue to change:

filters=30 //Formula calculation (5+categories)×3
activation=linear
[yolo]
mask = 3,4,5
anchors = 10,14, 23,27, 37,58, 81,82, 135,169, 344,319
classes =5

4. Training and testing

1. Terminal training

single card

Enter the darknet folder, enter the terminal, and execute the following command:

 sudo ./darknet detector train cfg/voc_blind.data cfg/yolov4_blind.cfg -map

Training result map
insert image description here

Doka

First check whether it is a multi-card gpu

#静态查询
nvidia-smi  

 #动态查询 每10秒检查gpu使用情况
watch -n 10 nvidia-smi

Check to see if those cards are not occupied. If they are occupied, it means that someone else is training or something, especially on the server. Don't mess with other people's things.

For example, I have four cards, of which 0,1 card GPU-Util accounts for 0%, then use 0,1 card to be idle.

sudo ./darknet detector train cfg/voc_blind.data cfg/yolov4_blind.cfg  -gpus 0,1 -map

Multiple cards are faster than single cards

insert image description here
Parameter explanation:

The table will display some information about the graphics card. The first line is the version information, the second line is the title bar, and the third line is the specific graphics card information. If there are multiple graphics cards, there will be multiple lines, and the information value of each line corresponds to Information about the corresponding position of the title bar.

* GPU：编号，这里是0，1，2,3

* Fan：风扇转速，在0到100%之间变动，第一个是32%

* Name：显卡名，这里都是GeForce

* Temp：显卡温度，第一个是55摄氏度

* Perf：性能状态，从P0到P12，P0性能最大，P12最小

* Persistence-M：持续模式的状态开关，该模式耗能大，但是启动新GPU应用时比较快，这里是off

* Pwr：能耗

* Bus-Id：GPU总线

* Disp.A：表示GPU的显示是否初始化

* Memory-Usage：显存使用率

* GPU-Util：GPU利用率，第一个是22%，第二个6%

* Compute M.：计算模式

One thing to note is that memory usage and GPU usage are two different things, similar to memory and CPU, and the usage of the two indicators does not necessarily correspond to each other.

error 1

CUDA Error: out of memory
Darknet error location: ./src/dark_cuda.c, check_error, line #69
CUDA Error: out of memory: No such file or directory

It means that the memory is insufficient, there are two possibilities: either your GPU memory is full, you can use nvidia-smi to see, or the batch and subdivisions values of the .cfg file may be too large, you can change it to 64>32>16> 8>1 and so on.

2. Save the model

The best model is the model that achieved the best performance during training. It is usually the model that achieves the best performance on the validation set.

The final model is the model saved at the end of the training process. It is usually the last model saved after training is complete.

The last model is the last saved model during training. It is usually the model saved at some point during the training process, and not necessarily the best performing model.

Therefore, these three models represent the models saved at different times during the training process, which can be used to compare the performance of the models and select the best model for inference.

Under dackup, where it thinks best is the best.
insert image description here

3. Test

1. Picture test

sudo ./darknet detector test cfg/voc_blind.data cfg/yolov4_blind.cfg backup/yolov4_blind_best.weights

After running, you will also need to enter the image path.
insert image description here

2.val verification

sudo ./darknet detector valid cfg/voc_blind.data cfg/yolov4_blind.cfg backup/yolov4_blind_best.weights

After running, the verification file of the verification set val will be generated.
insert image description here

3. Real-time detection

source code