Prepare for training yolov3 on the self-built data set (2): Raspberry Pi automatically collects pictures, tensorflow_label_tool downloads pictures, yolo_mark annotation and cleaning data (with python script)

Foreword

The previous article wrote how to download the data set on Google's open image v4 and save it in yolo format, then write it here. In order to confirm that there is no problem with the label, we can use yolo_mark to clean it again. And sometimes we may need to collect data by ourselves. Here I use the Raspberry Pi and a wide-angle camera to complete my automatic collection task. If we can, we can also crawl some data back online, here I used a small tool on github, the effect is not bad.

tool

raspberry pie
usb camera
yolo_mark
tensorflow_label_tool

Raspberry Pi automatic collection script

The Raspberry Pi adds a wide-angle camera. Because I want a larger scene, I chose a 170-degree (143-degree horizontal) wide-angle camera with only 2 million pixels, which caused a lot of loss in the final photo. Details. Install opencv on the Raspberry Pi, plug in the USB camera, and run the following program to automatically collect pictures. The pictures are placed in the current folder and named after the current system time. Unplug the camera when not collecting to exit the program.

import cv2
import datetime
cap = cv2.VideoCapture(0)
width = 1920
height = 1080
cap.set(cv2.CAP_PROP_FRAME_WIDTH,width)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT,height)
i = 1
while(cap.isOpened()):
    cv2.waitKey(5)
    i = i + 1
    ret, frame = cap.read()
    now_time = datetime.datetime.now()
    time_str = datetime.datetime.strftime(now_time,'%Y-%m-%d-%H:%M:%S')
    if frame is None:
        continue
    elif i > 15:
        i = 1
        cv2.imwrite(time_str+'.jpg' ,frame)
        cv2.imshow('cap',cv2.resize(frame,(480,270)))

cap.release()
cv2.destroyAllWindows()

tensorflow_label_tool tool download pictures

tensorflow_label_tool is an app running on windows. Select download, enter keywords, number of downloads, file name, and zoom width and height (when the value is 0, it means no zoom, the default is 300x300, which means that the image is filled or compressed to the black background without changing the original shape. 300x300) The result is placed in the download directory.

Tools in tensorflow implement cropping

Random cropping is a data enhancement method in tensorflow. The input image and the size you want to crop can get a specific output. Each crop will only get one picture, so I use multiple loops to get multiple cropped pictures. Although I release the variables after each big loop, as the input picture increases, there will still be insufficient memory, which has not been resolved. If a friend who knows the solution has the opportunity to see this, I hope I can also give me some advice. There are many other data enhancement methods, such as mirroring and changing colors. In addition, another purpose of cropping here is to reduce the density of the target in each picture, which is convenient for marking.

import tensorflow as tf
import cv2
import os
import glob 
import gc
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'

if __name__ == "__main__":
    WSI_MASK_PATH = 'G:\Yolo_mark-master'
    paths = glob.glob(os.path.join(WSI_MASK_PATH, '*.jpg'))
    paths.sort()
    for path in paths:
        img = cv2.imread(path)
        filename = os.path.basename(path)
        for cnt in range(4):
            crop_img = tf.random_crop(img,[550,550,3])
            sess = tf.InteractiveSession()
            cv2.imwrite("crop/" + str(cnt)+ "_"+ filename,crop_img.eval())
            img2 = cv2.imread("crop/" + str(cnt) + "_" + filename)
            cut_img = cv2.resize(img2,(608,608),interpolation = cv2.INTER_CUBIC)
            cv2.imwrite("cut/" + str(cnt) + "_" + filename,cut_img)
            sess.close()
            del crop_img,img2,cut_img,sess
            gc.collect()
            cv2.waitKey(100)   
        del filename,img,cnt,path
        gc.collect()
        cv2.waitKey(100)

yolo_mark marking and cleaning

The yolo_mark gadget used here is for marking, it supports windows and linux, and it can be used after compiling and generating an executable file. Before compiling, modify the obj.names file in a certain order to adapt it to your task requirements. Put the data you want to mark in the img folder, and then double-click yolo_mark.cmd to open yolo_mark to mark it. Enter "h" on the keyboard to view some operation commands. For example, "c" is to clear all the annotations of the current picture, and "r" is to clear the annotations of the selected target. Annotating a picture will correspondingly generate an annotated text file, and append a path to the train.txt file.

Clean with yolo_mark

There may be errors in the marked data. You can use yolo_mark to clean it again. By the way, all the paths are added to train.txt, and you can do both.