Summary of training ideas for lane line recognition algorithm by converting curvelanes data set to tusimple format [data cleaning]

This article contains content related to:

1. Data cleaning and sorting (specific resolution data does not meet the requirements, choose to eliminate, resize method under OpenCV)

2. Write data set format conversion script code

3. Possible problems

---

Foreword:

task background

At present, the ultra fast lane detection algorithm is used for training. This algorithm itself uses the culane data set and the tusimple data set for training. The configs part of the code dataloader provides the reading method of these two data sets, but considering the subsequent use Therefore, it is decided to separate the algorithm from the data set format conversion task, that is, the algorithm reads the data set in tusimple format, and converts all other data sets into Tusimple format, and the curvelanes data set is one of the few curve data sets, in which the curve data accounts for more than 60% of the total data, so the model can better recognize the curve and improve the recognition ability in multiple scenarios.

Ultra-Fast-Lane-Detection (ultra-fast lane line recognition algorithm) GitHub link

basic environment

1. free18.04
2. IDE：vscode
3. server:

---

提示：以下是本篇文章正文内容，下面案例可供参考

1. Tasks

Use the Windows file explorer to count the data quantity of the train and valid parts of the curvelanes data set. The number and proportion of pictures with different resolutions are as follows: the required

input picture is 1280×720, so the picture of 1570×660 needs to be to remove

2. Process analysis

1 idea

First convert the images and labels in the curvelanes data set to tusimple format, and then resize. During the process, you can choose to directly remove the pictures that do not meet the resolution requirements before conversion.

2 Necessity

1. Resizing the 1570*660 data directly will cause image size distortion, and the json in the corresponding labels file is in the form of dots, which will make it difficult to process the labels file. 2. For non-tusimple
data set, the importance of data cleaning is obviously self-evident
数据清洗：数据清洗是指发现并纠正数据文件中可识别的错误的最后一道程序，包括检查数据一致性，处理无效值和缺失值等。

3 implementation

3.1 Experiment with a small amount of data

First, in the curvelanes data set, select 20 2560×1440 pictures, 2 1560×660 pictures, 1 1280×720 pictures and their corresponding labels, and put them into two different folders (curvelanes data set is also processed in this way )

3.2 Two main points

1. When resizing a picture, you need to resize its labels file together, otherwise it is useless to just resize the picture, and the model cannot be trained if it is passed in; it is not recommended to use various software for resizing, because you don't know which one it uses resize method, and this software may not be able to recognize labels in PNG image format, and will prompt you "the file does not exist";
2. Eliminate pictures with a resolution of 1570*660. Since the resolution distribution of the data itself is very neat, use the traversal method to determine whether the picture is or not. widthAs heightlong as the width is 1570 or the height is 660, delete it directly.

3.3 Scheme

Finalize: Eliminate when converting to format, and then resize to prevent distortion

Part of the code:

(1) Eliminate pictures that do not meet the required resolution

for each_bmp in files:#遍历图片，进行筛选
    each_bmp_root =os.path.join(dir,each_bmp)#得到每个图片路径
    img = Image.open(each_bmp_root)#打开每个图片
    width = img.size[0]#获取图像的宽
    
    if width == 1570:#判断图片的宽为1570直接删除其本身及其label
        
        if os.path.exists(each_bmp_root):
            os.remove(each_bmp_root)
        else:
            print("The file does not exist")
     
        lablename= each_bmp.split("/")[-1]
        lablename = lablename.split(".")[0]
        # print(each_bmp_root)
        labelpath = os.path.join(mLabelPath,lablename)
        print(labelpath+'.lines.json')
        
        if os.path.exists(labelpath+'.lines.json'):
            os.remove(labelpath+'.lines.json')#remove的用法   chmod +x *可执行权限问题
        else:
            print("The file does not exist")
        
        print(each_bmp + ' ' + lablename+'.lines.json' + " deletefinished")

(2) Save the image after resize

 mPath = os.path.join(mSavePath, "20.jpg")#保存图片
    
    img = cv.resize(img,(1280,720))
    
    cv.imwrite(mPath,img) #openCV保存
    mString += mPath + " "
    mPath = os.path.join(mSavePath, "20.png")#保存标签
    
    img_gray = cv.resize(img_gray,(1280,720))
    
    cv.imwrite(mPath,img_gray)

3.4 Problem Solving

1. How to choose the resize method?

(1) Use the resize method in the OpenCV package

Resizing an image means changing dimensions, whether it's height or width alone, or both. You can also resize the image proportionally.
Syntax:
function prototype

cv2.resize(src, dsize[, dst[, fx[, fy[, interpolation]]]])

parameter:

parameter	describe
src	【Required】Original image
dsize	[Required] The required size of the output image
fx	[Optional] Scale factor along the horizontal axis
fy	[Optional] Scale factor along the vertical axis
interpolation	[Optional] Interpolation method

[Optional] Interpolation method
There are many interpolation methods:

cv.INTER	interpolation method
cv.INTER_NEAREST	nearest neighbor interpolation
cv.INTER_LINEAR	bilinear interpolation
cv.INTER_CUBIC	bilinear interpolation
cv.INTER_AREA	Resampling using pixel area relationships, which may be the method of choice for image decimation, as it provides moiré-free results. But when the image is zoomed, it's like INTER_NEAREST method

Usually, use cv.INTER_AREA for zooming out, and cv.INTER_CUBIC (slower) and cv.INTER_LINEAR (faster for better results) for scaling. By default, all scaling uses cv.INTER_LINEAR.

(2) Use the resize method in the PIL package

There are four interpolation methods for the parameters of the resize function in the Pillow package:

PIL.Image function
PIL.Image.NEAREST
PIL.Image.BILINEAR
PIL.Image.BICUBIC
PIL.Image.ANTIALIAS

When processing images, we try to use an image processing module from the beginning to the end, either all OpenCv, or Pillow or matplotlib, it is best not to open the image with OpenCv, crop the image and use Pillow and other operations, there will be some Unexpected error.

2. How to view the operation log of connecting to the remote server in vscode through setting?

Add the following parameters to the configuration file settings.json of VSCode:

"remote.SSH.showLoginTerminal": true,

Since the amount of data is generally relatively large when actually processing data, add this line to prevent the server from disconnecting to check whether the data has been processed, the disconnection time, etc., the blogger has not tried it, it is not necessary

3. The remove function uses

(1) Delete files

To remove a file, the os module must be imported and the os.remove() function run:
eg：

import os
if os.path.exists("demofile.txt"):
	os.remove("demofile.txt")
else:
	print("This file no found！")

(2) Delete folder

The remove function cannot delete a folder directly, and an error will be reported when using it to delete a folder:
because the function does not delete files in essence, remove just moves the pointer of the iterator forward to delete, and puts the undeleted elements in the linked list front, and returns an iterator pointing to the new supertail value. Since the remove() function is not a member, the length of the linked list cannot be adjusted. The remove() function is not a real deletion. To actually delete an element, you can use the erase() or resize() function. The usage is as follows:
eg：

string str1 = "Text with some   spaces";
str1.erase(std::remove(str1.begin(), str1.end(), ' '),  str1.end());  // "Textwithsomespaces"

To delete an entire folder, use the os.rmdir() method:
eg：

#只能删除空文件夹
import os
os.rmdir("folder")

4. Server-side execution error: No module named 'path':

sudo apt install path

5. Server-side execution error: No module named 'matplotlib':

pip install matplotlib

Summarize

After the code has passed the local experimental data test, the data processing is performed on the server side, and the resolution of 1570*660 is eliminated from the 20,000 images, which takes about 6 minutes. The conversion of the 20,000 images in the valid format takes about 65 minutes.

Reference link:

1. Resize the pictures and json files marked by labelme
2. Pytorch data enhancement, including simultaneous changes in image and label
3. The big pit of data set processing resize in image segmentation! ! !
4. Python uses Image to resize and save the image
5. Python script - data set acquisition: image crawling, invalid image removal, unified resolution, extraction of marked json and png
6. Python image resize() method summary
7. python-batch filter pictures-remove unwanted images
8. Deep learning voc dataset image resize
9. Comparison of various interpolation methods for OpenCV image scaling resize