Step by step teach you to use their own data sets mmdetection training at docker container

Mmdetection not described herein installation and configuration, using mmdetection simpler method is to use a container installed mmdetection the docker. Such a process eliminates the need for direct installation mmdetection, so focus on model training!

If you are not very familiar with the docker and mmdetection, search on your own, I will not repeat them.

Here attached mmdetection of GitHub address:

https://github.com/open-mmlab/mmdetection

0. preparatory

First, by default your computer has already done the preparatory work following these:

• Ubuntu 16.04 or higher
• GPU
• Installation cuda
• Installation cudnn
• Installation docker
• Install nvidia-docker

Of course, if you are connecting to a corporate or school server, and the server has been done to prepare the above points, you only need a Xshell Telnet server on the list.

1. Preparation: Download the docker mirror with mmdetection

First, we need to find a mirror configured docker good mmdetection environment. Can be performed on dockerhub with "mmdetection" as a keyword search, you can also use the direct command in the terminal in docker searchsearch:

$ docker search mmdetection

The results are shown as follows:

Here, we choose the first row of vistart / mmdetection mirror image docker download method is also very simple, the use of docker pullpull in the specified image from the image repository:

$ docker pull vistart/mmdetection

If the network is no problem, the download will be completed within a few minutes. After the download is complete, we can see if vistart / mmdetection image has been planted on the local mirror:

$ docker images

You can see vistart / mmdetection image has been successfully downloaded.

2. Create a new container containing the mmdetection

Contains mmdetection the image has been downloaded well, the next step is to create a new docker container for use:

$ docker run --runtime=nvidia --name mm_prj -i -t vistart/mmdetection /bin/bash

The above command to explain: --runtime=nvidiavery important, make GPU new docker container can use the host machine, this argument is without using the CPU default; --name mm_prjis a new docker container named, the name is mm_prj, readers can modify their own .

After the new interface container as follows:

At this point, called mm_prj container has been opened. You can see that the directory already contains mmdetection directory, it indicates that the docker mirror has been installed mmdetection.

supplement:

In addition, add some to exit the container, into the operation of the vessel.

Exit container:

# exit

View existing container:

$ docker ps -a

You can see, docker container named mm_prj has been in the container list.

Open container:

$ docker start mm_prj
$ docker exec -i -t mm_prj /bin/bash

3. Import your own data VOC

At this point, we need to own the data packaged into Pascal VOC format. Directory structure as follows:

VOCdevkit
--VOC2007
----Annotations
----ImageSets
------Main
----JEPGImages

Brief, which Annotations are stored in .xml files, JEPFImages is stored in .jpg picture.

According to this format after placing their own training data, segmentation requires training and testing data. Create a new file in the VOCdevkit test.py directory. test.py says:

import os
import random

trainval_percent = 0.8
train_percent = 0.8
xmlfilepath = 'Annotations'
txtsavepath = 'ImageSets\Main'
total_xml = os.listdir(xmlfilepath)

num = len(total_xml)
list = range(num)
tv = int(num * trainval_percent)
tr = int(tv * train_percent)
trainval = random.sample(list, tv)
train = random.sample(trainval, tr)

ftrainval = open('ImageSets/Main/trainval.txt', 'w')
ftest = open('ImageSets/Main/test.txt', 'w')
ftrain = open('ImageSets/Main/train.txt', 'w')
fval = open('ImageSets/Main/val.txt', 'w')

for i in list:
    name = total_xml[i][:-4] + '\n'
    if i in trainval:
        ftrainval.write(name)
        if i in train:
            ftrain.write(name)
        else:
            fval.write(name)
    else:
        ftest.write(name)

ftrainval.close()
ftrain.close()
fval.close()
ftest.close()

The above code dividing data set, trainval 80%, as the training set; Test 20%, as a test set.

Run test.py, will generate the following three files in VOCdevkit / ImageSets / Main directory:

Open the file you can see, trainval.txt sample index contains all training, test.txt contains all of the test sample index.

After their VOC data are made, is copied from the host (Ubuntu) to lower / mmdetection / data / directory:

$ docker cp VOCdevkit mm_prj:/mmdetection/data/

4. Modify class_names.py file

打开 /mmdetection/mmdet/core/evaluation/class_names.py 文件，修改 voc_classes 为将要训练的数据集的类别名称。如果不改的话，最后测试的结果的名称还会是’aeroplane’, ‘bicycle’, ‘bird’, ‘boat’,…这些。改完后如图：

5. 修改 voc.py 文件

打开 mmdetection/mmdet/datasets/voc.py 文件，修改 VOCDataset 的 CLASSES 为将要训练的数据集的类别名称。

如果只有一个类，要加上一个逗号，否则将会报错。

6. 修改配置文件

mmdetection 中提供了很多目标检测模型可供使用。例如，进入 /mmdetection/config/ 目录，就会看到很多模型：

根据我们选择使用的模型，修改相应的配置文件。本文我们使用的是FasterRCNN 模型，修改的是 faster_rcnn_r50_fpn_1x.py 文件。

6.1 修改 num_classes 变量

打开 faster_rcnn_r50_fpn_1x.py，将 num_classes 变量改为：类别数 + 1（例如我有 20 类，因此改为 21）：

6.2 修改 data_settings

因为 faster_rcnn_r50_fpn_1x.py 默认使用的是 coco 数据集格式，我们要对其修改成相应的 VOC 数据格式。修改后的内容如下图所示：

6.3 调整学习率

本文使用单 gpu 训练，修改 img_per_gpu = 2，workers_per_gpu = 0。

对学习率的调整，一般遵循下面的习惯：

• 8 gpus、imgs_per_gpu = 2：lr = 0.02；
• 2 gpus、imgs_per_gpu = 2 或 4 gpus、imgs_per_gpu = 1：lr = 0.005；
• 4 gpus、imgs_per_gpu = 2：lr = 0.01

这里，我们只使用单 gpu，且 img_per_gpu = 2，则设置 lr = 0.00125。

这里说一下 epoch 的选择，默认 total_epoch = 12，learning_policy 中，step = [8,11]。total_peoch 可以自行修改，若 total_epoch = 50，则 learning_policy 中，step 也相应修改，例如 step = [38,48]。

至此，配置文件已修改完毕。

7. 模型训练

模型训练非常简单，只需一行命令：

python3 ./tools/train.py ./configs/faster_rcnn_r50_fpn_1x.py

注意执行上面的命令是在 /mmdetection 目录下。

如果有多个 gpu，例如 0, 1 号 gpu 都可用，则可以全部用起来训练，命令如下：

CUDA_VISIBLE_DEVICES=0,1 python3 ./tools/train.py ./configs/faster_rcnn_r50_fpn_1x.py --gpus 2

上面的 --gpus 2 表示使用的 gpu 个数为 2。如果使用多块 gpu，注意修改学习率 lr。

然后，训练就开始了：

从打印出的信息中，可以看到当前的 epoch 和 loss 值。

每个 epoch 会生成一个模型，并自动保存在 /mmdetection/work_dirs/faster_rcnn_r50_fpn_1x/ 目录下。

训练完成之后，latest.pth 即 epoch_12.pth 就是最终的模型。

8. 模型测试，计算 mAP

下面我们将使用训练好的模型对测试集进行验证，并计算 mAP。

8.1 生成 pkl 文件

首先，生成 pkl 文件：

python3 ./tools/test.py ./configs/faster_rcnn_r50_fpn_1x.py ./work_dirs/faster_rcnn_r50_fpn_1x/latest.pth --out=result.pkl

8.2 计算测试集 mAP

对测试集计算 mAP，只需一行命令：

python3 ./tools/voc_eval.py result.pkl ./configs/faster_rcnn_r50_fpn_1x.py

计算结果如下：

图中可以看到，最后计算的 mAP = 0.978。