Recently, I am doing a final project, and I feel that the information on the Internet is not very complete. I wrote down my training process for friends who are doing this direction to learn. Only simple recurrence, the rest has not been explored.
1. Dataset and data preprocessing
The first is the data set: the data set I use is the data set released by NWPU, NWPU VHR-10
Link: https://pan.baidu.com/s/1vfhDU2ORWUpL-aGM1PllGw
Extraction code: d5au
The Xigongda dataset has ten categories and three folders, namely positive image set (650 pictures), negative image set (150 pictures) and ground truth (650 txt files).
The images in the negative image set have no corresponding ground objects,
the images in the positive image set include one or more corresponding ground objects, and
the txt in the ground truth corresponds to the circumscribed rectangle of the ground objects in the positive image set
Image information:
- image size: 500~1100 * 500~1100 * 3
- image number: 800
- object number: 3,651
Labeling information:
10 categories, a total of 3,651 objects;
- airplane
- ship
- storage tank, storage tank
- baseball diamond, baseball field
- tennis court
- basketball court, basketball court
- ground track field, ground track field
- harbor
- bridge
- vehicle
The first is to convert the NWPU VHR-10 into a form similar to the voc dataset and then convert it to the yolo format.
The voc file format includes file annotations to store image information, images to store training images, and ImageSets to store image training information.
Reorder the pictures in the negative image set and positive image set to 000001.jpg-000800.jpg, the pictures in the positive image set are 000001.jpg-000650.jpg, and the pictures in the negative image set are 000651.jpg-000800.jpg , and then stored in the image folder.
An xml file with the same format as the annotation information xml file in the annotations folder of the dataset. Because the pictures in the negative image set have no corresponding label information, only the xml file containing the picture size is generated.
The ImageSets folder is divided into four files: train.txt, val.txt, trainval.txt, and test.txt. train represents training set information, val represents verification set information, trainval represents the combined data information of training set and verification set, and test Represents the test set information.
Then convert the voc format to the yolo format, generate a new labelsfolder in the directory, import the data set path into the txt file, extract and convert each xml annotation information into the txt format, and each image corresponds to a txt file.
Attach my program, in which main.py, 1.py, 2.py, 3.py run in order , because the latter one may use the address of the thing generated by the previous one , just change the address to your own address .
Link: https://pan.baidu.com/s/1C04y16LIdyby5vhNEzsYSA
Extraction code: iz0r
There is something wrong. Change the line in 3.py to the sentence below. There is one bridge class missing, one missing between harbor and vehicle, and tennis is misspelled.
classes = ['airplane','ship','storage tank','baseball diamond','tennis court','basketball court','ground track field','harbor','bridge','vehicle'] # class names! ! ! Lots of addresses ! ! ! And it must be run in order. If you are not sure whether the folder can be generated , you can create it yourself in advance, and then run it directly. You don’t need to change anything except the address! ! ! ! !
2. Environment construction
First of all, attach the official source code of yolov5, you can download it on Github by yourself, search yolov5 directly on Github, and download the one with the most likes. I have downloaded it to my network disk, here is my network disk link.
Link: https://pan.baidu.com/s/1BENXIeyKgjwyAhnXA_n9Uw
Extraction code: 3r9o
yolov5 needs pytorch and cuda to run, because I have run some neural network deep learning algorithms before, so my environment has already been set up. In addition, if you want to improve the running speed, you need to download the GPU, and the running speed with only the cpu is too fast. Slow, but it works. The best python version is 3.7 or 3.8. If it is too high, an error will be reported. You can lower the python version in advance before running. I am using conda version 3.8.
If it is pure white, read this blog, installation environment
Record the whole process of configuring the Yolov5 environment using Anaconda and Pycharm
After the environment is set up, the next step is to install the required libraries
pip install -r requirements.txtyolov5 comes with a requirements.txt file, which is all the libraries that can be used at runtime. Just run it in the terminal according to the above statement. Although it will not be completed smoothly under normal circumstances, if you can’t install it, you can manually pip it yourself. Win+r enter cmd to enter the command prompt window, enter pip list, you can see all libraries and versions, compare the install of the version given in requirements.txt, and the uninstall of the uninstall.
3. Run the program
First, you need to create a mydata.yaml under the data folder. Like this, mydata.yaml is a subfile of data.
The mydata.yaml program, the yaml format is very strict, there must be a space after the colon except for the address, otherwise an error will be reported (because I changed this space for two hours), and more ! ! !
The addresses of val, train, and test are the addresses generated in the last step of our data preprocessing.
Here, it is the same level file as the folder. As for the two .cache files, don’t worry about it. This is generated during training. It’s normal that there is no one now, so don’t make a mistake.
This is a program, I suggest you just copy and change the address directly. Briefly, the first three lines are the address, the second is the category, and the last one is the number of categories, which can also be modified according to your own data set.
val: G:/csdn/yolov5/nwpu vhr-10/val.txt
train: G:/csdn/yolov5/nwpu vhr-10/train.txt
test: G:/csdn/yolov5/nwpu vhr-10/test.txt
names:
- airplane
- ship
- storage tank
- baseball diamond
- tennis court
- basketball court
- ground track field
- harbor
- bridge
- vehicle
nc: 10Then modify the main program, train.py, under the utils file
The main parts that need to be modified are these five. If you are using the source code, you only need to change the yaml file on the third line to Mydata. The .pt file in the first line is a weight file, which can be changed, and will be mentioned later.
The next step is to modify the yaml file. In the second line, we use yolo5s.yaml, so modify it in model/yolov5s.yaml.
Just change the nc number to the same as above, because the Xigongda dataset has only ten categories, so my nc is 10.
Then you can run it.
If there is a problem of not being able to connect to the network or the other party not responding, it will download the weight file by itself during operation, which is the pt file of yolov5s.pt in the first line above. It doesn’t matter if there is a problem, you can Just download the weight file and put it in a folder.
Link: https://pan.baidu.com/s/1p7_v_Wlicw_7oBUv-QNv_w
Extraction code: uarf
This is the link to the weight file, put it in the root directory, that is to say, it is the same level file as the model, parallel! !
The weight file looks like this, the bigger the memory, the bigger the weight, the better the training effect, not absolute! ! ! Be careful of overfitting, and the running speed zun is very slow. I used the Xigong big data set running in 5s, 100 epochs, ran for ten hours, changed the weight file, 5x, and could not finish an epoch in an hour.
This is no problem, the result of running will be saved in runs\train\exp
Four, detect
Also in the utils folder, modify these three files, the first is the weight file we use, the second is the address of the picture we need to detect (be sure to replace it with your own picture!!!), the third Just the old rules. Then it can be run, the generated result is in runs\test\exp, I forgot to take a screenshot.
5. Change the weight file
The first step is to modify the five boxes in the train, and change 5s to something else. It is recommended to change the epoch and batch-size to a smaller size if a large weight file is used, otherwise it will time out. Second, no, change it accordingly The nc number in the pt file, you have an impression, and then you can run it. If an error is reported, you can directly use a line of code to run it in the terminal. (Because I reported an inexplicable error when I ran it myself, so I used this method, it is relatively simple, no need to change the source code, nothing needs to be changed.
python train.py --batch-size 2 --epochs 100 --data G:\bishe\bishe1\yolov5-master\data\mydata.yaml --weights D:\BaiduNetdiskDownload\yolov5-weights\yolov5x.ptFour elements, batchsize, epochs, the address of mydata.yaml and the address of the weight file.pt. This method does not need to change the code, and does not need to move anything. It doesn’t matter what you change the few boxes I just circled, just this line will do.
Similarly, detect can also use this method.
python .\detect.py --source C:\Users\Administrator\PycharmProjects\yolov5\imgs\valid\images --weights C:\Users\Administrator\PycharmProjects\yolov5\runs\train\exp8\weights\best.ptTwo elements, one is the address of the picture you need to test, the other is the pt file, this file is what we trained, in runs\train\exp\weights\best.pt
This is the same, you don't need to change the source code, and you don't need to change the source code frame to what kind of frame, you can use this method.
Put a picture of detect.
This is the whole process of using yolov5 to train the Xigong big data set. I figured it out a little bit. If there is something wrong, please leave a message and correct me. If you have any questions, you can also private message me. I will answer yes, no I can only feel sorry for it, and I will post it if there are improvements in the future. thank you all! ! !