YOLO V5 training custom data set

Please note: Each file path cannot contain Chinese characters.

1. Environment setup

1. Clone the repository from github to local
   https://github.com/ultralytics/yolov5
   VSCode. Cloning the repository requires Git to be installed.
   
   

2. Build the Yolo5 environment.
   The external libraries in the requirements.txt file under the project that the Yolo environment depends on
   can be installed individually through pip, or they can be installed in batches by running the requirements.txt using CMD.
   
   
   
   

C:\Users\viarondu>cd C:\Users\viarondu\Desktop\01\YOLO\yolov5
C:\Users\viarondu\Desktop\01\YOLO\yolov5>pip install -r requirements.txt

2. Image annotation

Labeling scheme 1 uses labelImg for labeling (recommended)

1. Double-click to run the specified software
   

2. Set folder path
   

3. Turn on auto-save settings
   
   

4. Frame objects for labeling
   

5. Create the following folder under the data file of yolo5:
   
   images stores all annotated images.
   JPEGImages stores the same content as the images file. It also stores all annotated images.
   Annotations stores annotated xml files. Copy the annotated xml file generated under the labelImg export file to in this folder
   
   

Marking scheme 2 uses VoTT

Files exported using VoTT annotation may report errors in Yolo

1. Download VoTT (Visual Object Tagging Tool).

2. Open VoTT and select "New Project".
   
   

3. In Project Settings, change the Display Name to the project name.

4. Change the security token to generate a new security token.

5. Next to Source connection, select Add connection.

6. In Connection Settings, set the image folder.
   

7. In Project Settings, change the Source Connection to the connection you just created.

8. Also change the "Target Connection" to the connection you just created, it should now look like the following screenshot:
   

9. Select Save Project.
   Add markers and label images
   You should now see a window with a preview of all training images on the left, a preview of the selected image in the middle, and a labeled column on the right.

10. Select the first (plus sign) icon in the Markers toolbar to add a new marker.

11. Name the sign "Stop-Sign" and press Enter on your keyboard.

12. Click and drag your mouse to draw a rectangle around each stop sign in the image. If you can't draw a rectangle with your cursor, try selecting the Draw Rectangle tool from the toolbar at the top, or use the R keyboard shortcut.

13. After drawing the rectangle, select the Stop-Sign mark created in the previous step and add this mark to the bounding box.

14. Repeat this process by clicking on the preview image for the next image in the dataset.

15. Continue to standardize each stop sign in each image.

Export Pascal VOC
After you have labeled all training images, you can export the file so that Model Builder can use it for training.

1. Select the fourth icon in the left toolbar (the one with the diagonal arrow in the box) to go to Export Settings.

2. Leave "Provider" as Pascal VOC.

3. Select Save Export Settings.
   

4. Return to the Markup Editor (the second icon in the left toolbar shaped like a ribbon). In the top toolbar, select the Export Project icon (the last icon, shaped like a box containing an arrow), or use the keyboard shortcut Ctrl+E.
   This export operation will create a new folder named *****-PascalVOC-export under the image resource folder, and will generate the following annotation file in this new folder:
   

5. The VoTT exported file is as follows:
   
   copy it to the data folder of the YOLO project, and create a new images and labels folder
   images to store the training photos. You need to save a copy of the training photos under this file.
   
   

3. YOLO project annotation configuration file generation

1. Generate Yolo annotation configuration file

Create a trans_txt.py file in the root directory of yolov5. The file code is as follows:

import os
import random
trainval_percent = 0.1
train_percent = 0.9
xmlfilepath = 'data/Annotations'
txtsavepath = 'data/ImageSets'
total_xml = os.listdir(xmlfilepath)
num = len(total_xml)
list = range(num)
tv = int(num * trainval_percent)
tr = int(tv * train_percent)
trainval = random.sample(list, tv)
train = random.sample(trainval, tr)
ftrainval = open('data/ImageSets/trainval.txt', 'w')
ftest = open('data/ImageSets/test.txt', 'w')
ftrain = open('data/ImageSets/train.txt', 'w')
fval = open('data/ImageSets/val.txt', 'w')
for i in list:
    name = total_xml[i][:-4] + '\n'
if i in trainval:
    ftrainval.write(name)
if i in train:
    ftest.write(name)
else:
fval.write(name)
else:
ftrain.write(name)
ftrainval.close()
ftrain.close()
fval.close()
ftest.close()

2. Continue to create the voc_label.py file, the code is as follows:

import xml.etree.ElementTree as ET
import pickle
import os
from os import listdir, getcwd
from os.path import join
sets = ['train', 'test','val']

# classes数组输入打的标签种类
classes = ['heros','soliders','tower','red_buff','blue_buff','pig','wolf','bird','lizards']
def convert(size, box):
    dw = 1. / size[0]
    dh = 1. / size[1]
    x = (box[0] + box[1]) / 2.0
    y = (box[2] + box[3]) / 2.0
    w = box[1] - box[0]
    h = box[3] - box[2]
    x = x * dw
    w = w * dw
    y = y * dh
    h = h * dh
    return (x, y, w, h)
def convert_annotation(image_id):
    in_file = open('data/Annotations/%s.xml' % (image_id))
    out_file = open('data/labels/%s.txt' % (image_id), 'w')
    tree = ET.parse(in_file)
    root = tree.getroot()
    size = root.find('size')
    w = int(size.find('width').text)
    h = int(size.find('height').text)
    for obj in root.iter('object'):
        difficult = obj.find('difficult').text
        cls = obj.find('name').text
        if cls not in classes or int(difficult) == 1:
            continue
        cls_id = classes.index(cls)
        xmlbox = obj.find('bndbox')
        b = (float(xmlbox.find('xmin').text), float(xmlbox.find('xmax').text), float(xmlbox.find('ymin').text),
             float(xmlbox.find('ymax').text))
        bb = convert((w, h), b)
        out_file.write(str(cls_id) + " " + " ".join([str(a) for a in bb]) + '\n')
wd = getcwd()
print(wd)
for image_set in sets:
    if not os.path.exists('data/labels/'):
        os.makedirs('data/labels/')
    image_ids = open('data/ImageSets/%s.txt' % (image_set)).read().strip().split()
    list_file = open('data/%s.txt' % (image_set), 'w')
    for image_id in image_ids:
        list_file.write('data/images/%s.jpg\n' % (image_id))
        convert_annotation(image_id)
    list_file.close()

This file needs to be modified. The statement
classes = ['heros','soliders','tower','red_buff','blue_buff','pig','wolf','bird','lizards'] in the code requires Modify according to the categories when you label, write several categories;
run the two files trans_txt.py and voc_label.py in sequence, and the txt file will be
generated under labels: (1) The txt file will be generated under labels (displaying the specific annotation data of the data set) )

(2) Four txt files are generated under ImageSets

(3) Three txt files are generated under data (with the path of the picture)

4. YOLO training parameter settings

1. Set the coco.yaml file under the data file

Parameter Description:

path: ../datasets/coco  # dataset root dir  datasets根路径，这里不使用，不做修改
# 训练图片标注文件
train: C:/Users/viarondu/Desktop/01/YOLO/yolov5/data/train.txt  # train images (relative to 'path') 118287 images
# 验证图片标注文件
val: C:/Users/viarondu/Desktop/01/YOLO/yolov5/data/val.txt  # val images (relative to 'path') 5000 images
#测试文件标注文件
test: C:/Users/viarondu/Desktop/01/YOLO/yolov5/data/test.txt  # 20288 of 40670 images, submit to https://competitions.codalab.org/competitions/20794


# Classes  对象标签，有几类就填几类，ID从0开始
names:
  0: Stop-Sign

2. Modification of models model files

In this case, the Yolov5m network structure is selected. This network structure takes into account both speed and accuracy.
Find yolov5m.yaml under the models folder, and change nc to the number of recognized object types.

3. Weight file configuration

Create a folder named weights under the yolov5 folder, and then place download_weights.sh under the data/scripts file under the weights file;
weight file download address:

Store the downloaded weight file under the weights file:

4. Set the train.py file

1.–weights: weight file address
2.–cfg: model file address, the path is yolov5s.yaml under the models folder, if used For other models, just change it to the yaml file of the corresponding model under the models file.
3.–data: For this path, choose the yaml file we configured in the fourth step. Mine is coco_Aron.yaml
4.–epochs: How many times will the entire data set be iterated during the training process
? 5. –batch-size: How many pictures are viewed at one time before the weight is updated. It is best to be a multiple of 2. If it is set to -1, it will be automatically adjusted according to the computer configuration.

5. Training and testing results

After the setup is complete, run train.py

After the training is completed, the corresponding weight file


test results will be generated in the yolov5\runs\train\exp path:
Set

the key parameter description of the detect.py file:
–weights: the address of the weight file exported after the training is completed
–source: the test file path
–conf -thres: score threshold
–max-det: the maximum number of objects found in a single image
Run the detect.py file:
the final recognition result will be generated under the yolov5\runs\detect\exp path