Preliminary summary of the safety helmet detection system

table of Contents

Data set part

Backend

Front end

Front and back connection

Data set part:

In this project, we use the traditional open source dataset SHWD (Safety helmet (hardhat) wearing detect dataset)

The data set label is hat, person

Some thoughts on the data set

The collection and labeling of the data set have a very important influence on the accuracy of the subsequent program design and prediction model

1. First of all, I am not satisfied with the labels of the data set for the time being. It is more reasonable to set three labels in my thoughts:
for one, we should draw a real frame for the whole body and mark the person, and draw for the head The real box marks hat, head or hat, no hat. For the time being, we are only predicting the head. If we wear a helmet, we will display the label hat, and if we don’t wear a hat, we will display person. In this regard, the following drawbacks are first discovered:
1) For yolo's target detection, if the person's head is smaller when the figure is smaller, the head scan is not accurate, and the whole body scan is more accurate at this time. Recognize people, and if the figure shows the back of the head, the recognition effect is particularly bad, and the overall scan can greatly reduce the occurrence of such problems.

2) For the following target tracking problem, using the whole as a target is more accurate and easier to handle than using the head. Now if you choose to track the person, deepsort will not track the person wearing the helmet. It does not meet our design expectations. So for the time being, our approach is to track and count both the hat and the person at the same time. It is barely qualified, but there is still a real situation. Some of the problems in the video, such as the fact that there is only one helmet in the video, no one wears it, and the technology does not meet our expectations. For this problem, we temporarily thought of two methods:

Ⅰ. The
first is the improvement to the data set we mentioned above . If the data set is improved in this way, we can use the tag person that recognizes the entire body as the tracking object, thereby avoiding this problem. And the display is clearer, otherwise the tracking frame and the identification frame will only appear on the head display, which will make it unclear, and it will be clearer after the change.
Ⅱ.
If we keep the current data set, we can use the trained yolo4 model that is trained to recognize 20 kinds of objects. We are importing two yolo models. One model parameter is based on the data set we are not satisfied with. A good model for recognizing heads and hats is used for yolo's target detection, and another model is loaded into the downloaded model to recognize the entire human body as an object for target tracking. This can also achieve the above content. We need to use sess.run() to run two models in turn from the picture of each frame as the data, predict the objects on the picture, and then separate the frames. But for the time being, we can't use sess.run() to run different graphs with the same data.

2. For the problem that the definition of a given video is too low, we can also use the expansion of the data set to improve the model's ability to recognize fuzzy pictures. We can find some fuzzy hard hat pictures, or find a fuzzy hard hat. Extract the picture frame by frame, then label the picture, and finally train. In my opinion, this method is beneficial to our model's ability to recognize objects in blurred videos.

3. We must consider the problem of identifying safety helmets comprehensively, and consider the prone conditions of safety helmets to fill the data set, so as to improve the recognition effect of our model against various extreme situations.
1) First, we should refer to national security The helmet standard requires that we have a basic understanding of basic helmet styles, colors, and shapes. If our data set covers all possible helmets, this will strengthen our professionalism in the field of identifying helmets.

2) Second, we need to consider the arbitrariness of people's activities, which may be easy to identify for the front, but we also need to take into account the side and back of people and helmets, which will help improve the accuracy of model recognition.

4. Can we use the data set to achieve more advanced functions? Can we train not only whether we need to detect whether to wear a helmet and people, but also to identify common items on the construction site. When someone is not wearing a helmet, we It can identify the objects near the person, and prompts that the person near the object does not wear a helmet, please wear it in time to avoid danger.

To sum up, more than half of the whole project gave me a big feeling that the quality of the data set directly affects the quality of our entire system and the subsequent series of work, so we continue to collect a wide range of data sets throughout the process of the software cup. Training a better model should be our constant task.

Backend

In my opinion, the back-end part is divided into two parts

The first one is YOLO

The second part is deepsort

The last is the connection between the two.

YOLO

The training file of the model is in G:/software cup/project

Tips for setting

Under train.py and train_eager.py files:
1. The mosaic parameter can be used to control whether to realize Mosaic data enhancement.
2. Cosine_scheduler can be used to control whether to use learning rate cosine annealing decay.
3. label_smoothing can be used to control whether Label Smoothing is smooth.

Under the train_eager.py file:
1. The regularization parameter can be used to control whether to achieve regularization loss.

file download

The yolo4_weights.h5 required for training can be downloaded from Baidu SkyDrive.
Link: https://pan.baidu.com/s/1DNv71lDkeWff2BmnVpgHeg Extraction code: myz8
yolo4_weights.h5 is the weight of the coco data set.
yolo4_voc_weights.h5 is the weight of the voc data set.

Prediction step

1. Use pre-training weights

a. After downloading the library, unzip it, download yolo4_weights.h5 or yolo4_voc_weights.h5 from Baidu network disk, put it in model_data, run predict.py, and enter

img/street.jpg

The forecast can be completed.
b. Use video.py to perform camera detection.

2. Use self-trained weights

a. Follow the training steps to train.
b. In the yolo.py file, modify model_path and classes_path in the following parts to make them correspond to the trained files; model_path corresponds to the weight file under the logs folder, and classes_path is the class corresponding to model_path .

_defaults = {
    
    
    "model_path": 'model_data/yolo4_weight.h5',
    "anchors_path": 'model_data/yolo_anchors.txt',
    "classes_path": 'model_data/coco_classes.txt,
    "score" : 0.5,
    "iou" : 0.3,
    # 显存比较小可以使用416x416
    # 显存比较大可以使用608x608
    "model_image_size" : (416, 416)
}

c. Run predict.py and enter

img/street.jpg

The forecast can be completed.
d. Use video.py to perform camera detection.

Training steps

1. This article uses VOC format for training.
2. Before training, put the label file in the Annotation under the VOC2007 folder under the VOCdevkit folder.
3. Before training, place the picture files in the JPEGImages under the VOC2007 folder under the VOCdevkit folder.
4. Use the voc2yolo4.py file to generate the corresponding txt before training.
5. Run voc_annotation.py in the root directory again, you need to change the classes to your own classes before running. Be careful not to use Chinese labels, and no spaces in the folder!

classes = ["aeroplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair", "cow", "diningtable", "dog", "horse", "motorbike", "person", "pottedplant", "sheep", "sofa", "train", "tvmonitor"]

6. At this time, the corresponding 2007_train.txt will be generated, and each line corresponds to the position of the picture and the position of the real frame and the category in the corresponding real frame (and the category is numbered by a number. For example, there are two categories that may use the number 0. Instead, use the number 1 for person).

7. Before training, you need to create a new txt document under model_data, enter the classes to be classified in the document , and point classes_path to the file in train.py . The example is as follows:

classes_path = 'model_data/new_classes.txt'    

The content of the model_data/new_classes.txt file is:

cat
dog
...

8. Run train.py to start training.

The specific internal algorithm that deepsort is called directly is unclear.

Combination of the two (yolo+deepsort)

The first part loads the trained model

import colorsys
import numpy as np
# from keras import backend as K
import tensorflow as tf
tf.compat.v1.disable_v2_behavior()
tf.compat.v1.disable_eager_execution()
from timeit import default_timer as timer
from PIL import Image, ImageFont, ImageDraw
from tensorflow.keras import backend as K
from tensorflow.keras.models import load_model
from tensorflow.keras.layers import Input
# from tensorflow.keras.models import load_weights
from yolo4.model import yolo_eval, Mish
from yolo4.utils import letterbox_image
import os
from keras.utils import multi_gpu_model
from yolo4.model import yolo4_body
os.environ["CUDA_VISIBLE_DEVICES"] = "1"

class YOLO(object):
    def __init__(self):
        #这一模型载入是为了实现deepsort和yolo识别模型不同权重，而暂时未实现
        self.weights_path = './model_data/yolo4_weight.h5'
        #载入训练好的模型
        self.model_path = './model_data/yolo4_model.h5'
        #载入真实框尺寸
        self.anchors_path = './model_data/yolo_anchors.txt'
        #载入自定义识别类别
        self.classes_path = './model_data/coco_classes01.txt'
        self.gpu_num = 1
        self.score = 0.5
        self.iou = 0.5
        self.eager = False
        #读取txt设定的类别读取并组织成列表形式
        self.class_names = self._get_class()
        #获得真实框尺寸组织成列表
        self.anchors = self._get_anchors()
        #获得跑图工具用在跑载入的模型
        self.sess = tf.compat.v1.keras.backend.get_session()
    
        self.model_image_size = (608, 608)  # fixed size or (None, None)
        self.is_fixed_size = self.model_image_size != (None, None)
        self.boxes, self.scores, self.classes = self.generate()

    def _get_class(self):
        classes_path = os.path.expanduser(self.classes_path)
        with open(classes_path) as f:
            class_names = f.readlines()
        class_names = [c.strip() for c in class_names]
        return class_names

    def _get_anchors(self):
        anchors_path = os.path.expanduser(self.anchors_path)
        with open(anchors_path) as f:
            anchors = f.readline()
            anchors = [float(x) for x in anchors.split(',')]
            anchors = np.array(anchors).reshape(-1, 2)
        return anchors

    def generate(self):
        model_path = os.path.expanduser(self.model_path)
        assert model_path.endswith('.h5'), 'Keras model or weights must be a .h5 file.'
        # self.yolo_model= load_model(model_path, custom_objects={'Mish': Mish}, compile=False)
        # # self.yolo_model.load_weights(self.weights_path)
        num_anchors = len(self.anchors)
        num_classes = len(self.class_names)
        print(num_classes)
        #载入yolo4模型，如果你之前训练模型的时候存的是模型则可以直接使用load载但是如果你只
        是保存参数，则需要将原来模型实例化一个后再载入参数，load_weights。
        self.yolo_model = yolo4_body(Input(shape=(None, None, 3)), num_anchors // 3, num_classes)
        self.yolo_model.load_weights(self.weights_path)
        # self.dect_model = load_model(model_path, custom_objects={'Mish': Mish}, compile=False)
        print('{} model, anchors, and classes loaded.'.format(model_path))

        # Generate colors for drawing bounding boxes.
        hsv_tuples = [(x / len(self.class_names), 1., 1.)
                      for x in range(len(self.class_names))]
        self.colors = list(map(lambda x: colorsys.hsv_to_rgb(*x), hsv_tuples))
        self.colors = list(
            map(lambda x: (int(x[0] * 255), int(x[1] * 255), int(x[2] * 255)),
                self.colors))
        np.random.seed(10101)  # Fixed seed for consistent colors across runs.
        np.random.shuffle(self.colors)  # Shuffle colors to decorrelate adjacent classes.
        np.random.seed(None)  # Reset seed to default.

        # Generate output tensor targets for filtered bounding boxes.
        self.input_image_shape = K.placeholder(shape=(2, ))
        if self.gpu_num>=2:
            self.yolo_model = multi_gpu_model(self.yolo_model, gpus=self.gpu_num)
        boxes, scores, classes = yolo_eval(self.yolo_model.output, self.anchors,
                len(self.class_names), self.input_image_shape,
                score_threshold=self.score, iou_threshold=self.iou)
        return boxes, scores, classes
        
        
    #该函数会在main函数当中调用，我们会在主函数中将输入的视频提取出每一帧为图片
    作为参数输入该函数当中，我们使用训练好的模型，将图片作为输入数据，对图中的
    目标物体做预测，产生以下信息，预测框位置，预测类别，预测置信度，根据这些数
    我们对图片进画框并目标跟踪
    
    
    def detect_image(self, image):
        start = timer()
        #
        # 调整图片使其符合输入要求
        new_image_size = (self.model_image_size[1], self.model_image_size[0])
        boxed_image = letterbox_image(image, new_image_size)
        image_data = np.array(boxed_image, dtype='float32')
        image_data /= 255.
        image_data = np.expand_dims(image_data, 0)  # Add batch dimension.

        if self.eager:
            # 预测结果
            input_image_shape = np.expand_dims(np.array([image.size[1], image.size[0]], dtype='float32'), 0)
            out_boxes, out_scores, out_classes = self.yolo_model.predict([image_data, input_image_shape])
        else:
        #使用模型进行预测
            out_boxes, out_scores, out_classes = self.sess.run(
                        [self.boxes, self.scores, self.classes],
                        feed_dict={
                            self.yolo_model.input: image_data,
                            self.input_image_shape: [image.size[1], image.size[0]],
                            K.learning_phase(): 0
                    })

        print('Found {} boxes for {}'.format(len(out_boxes), 'img'))
        # 设置字体
        font = ImageFont.truetype(font='font/simhei.ttf',
                                  size=np.floor(3e-2 * image.size[1] + 0.5).astype('int32'))
        thickness = (image.size[0] + image.size[1]) // 300

        small_pic = []
        for i, c in list(enumerate(out_classes)):
            predicted_class = self.class_names[c]
            box = out_boxes[i]
            score = out_scores[i]

            top, left, bottom, right = box
            top = top - 5
            left = left - 5
            bottom = bottom + 5
            right = right + 5
            top = max(0, np.floor(top + 0.5).astype('int32'))
            left = max(0, np.floor(left + 0.5).astype('int32'))
            bottom = min(image.size[1], np.floor(bottom + 0.5).astype('int32'))
            right = min(image.size[0], np.floor(right + 0.5).astype('int32'))

            # 画框框
            label = '{} {:.2f}'.format(predicted_class, score)
            draw = ImageDraw.Draw(image)
            label_size = draw.textsize(label, font)
            label = label.encode('utf-8')
            print(label)

            if top - label_size[1] >= 0:
                text_origin = np.array([left, top - label_size[1]])
            else:
                text_origin = np.array([left, top + 1])

            for i in range(thickness):
                draw.rectangle(
                    [left + i, top + i, right - i, bottom - i],
                    outline=self.colors[c])
            draw.rectangle(
                [tuple(text_origin), tuple(text_origin + label_size)],
                fill=self.colors[c])
            draw.text(text_origin, str(label, 'UTF-8'), fill=(0, 0, 0), font=font)
            del draw
        end = timer()
        print(end - start)

        #进行追踪
        return_boxes = []
        return_scores = []
        return_class_names = []
        count = 0
        # with graph.as_default():
        for i, c in reversed(list(enumerate(out_classes))):
            predicted_class = self.class_names[c]
            # if predicted_class != 'person':  # 确定追踪类别，追啥就是啥
            #     continue
            #显示当前屏幕内未带人数
            if predicted_class == 'person':  # Modify to detect other classes.
                count = count + 1
            box = out_boxes[i]
            score = out_scores[i]
            x = int(box[1])
            y = int(box[0])
            w = int(box[3] - box[1])
            h = int(box[2] - box[0])
            if x < 0:
                w = w + x
                x = 0
            if y < 0:
                h = h + y
                y = 0
            return_boxes.append([x, y, w, h])
            return_scores.append(score)
            return_class_names.append(predicted_class)

        return return_boxes, return_scores, return_class_names,image,count

    def close_session(self):
        self.sess.close()

The second part of the main function part

from __future__ import division, print_function, absolute_import
import sys
#sys.path.remove('/opt/ros/kinetic/lib/python2.7/dist-packages')
import os
import datetime
from timeit import time
import warnings
import cv2
import numpy as np
import argparse
from PIL import Image
from yolo import YOLO

from deep_sort import preprocessing
from deep_sort import nn_matching
from deep_sort.detection import Detection
from deep_sort.tracker import Tracker
from tools import generate_detections as gdet
from deep_sort.detection import Detection as ddet
from collections import deque
from keras import backend
import tensorflow as tf
from tensorflow.compat.v1 import InteractiveSession
# config = tf.ConfigProto()
config = tf.compat.v1.ConfigProto()
config.gpu_options.allow_growth = True
session = InteractiveSession(config=config)
from PyQt5 import QtCore, QtGui, QtWidgets
from PyQt5.QtGui import *
from PyQt5.QtCore import *
from PyQt5.QtWidgets import QInputDialog, QLineEdit, QDialog
from PyQt5.QtWidgets import QApplication
from probar import *
ap = argparse.ArgumentParser()
ap.add_argument("-i", "--input",help="path to input video", default = "./test_video/test.mp4")
ap.add_argument("-c", "--class",help="name of class", default = "person")
args = vars(ap.parse_args())

pts = [deque(maxlen=30) for _ in range(9999)]
warnings.filterwarnings('ignore')

# initialize a list of colors to represent each possible class label
np.random.seed(100)
COLORS = np.random.randint(0, 255, size=(200, 3),
	dtype="uint8")
#list = [[] for _ in range(100)]

def mainback(self,input,yolo):
    # QtCore.QCoreApplication.setAttribute(QtCore.Qt.AA_EnableHighDpiScaling)
    # app = QApplication(sys.argv)
    #展开等待界面
    wait()
    #开始计时
    start = time.time()
    max_cosine_distance = 0.3
    nn_budget = None
    nms_max_overlap = 1.0

    counter = []
    #deep_sort
    #载入deepsort所用模型
    model_filename = 'model_data/market1501.pb'
    encoder = gdet.create_box_encoder(model_filename,batch_size=1)
    
    #设置查找目标
    find_objects = ['person']
    #调用追踪函数
    metric = nn_matching.NearestNeighborDistanceMetric("cosine", max_cosine_distance, nn_budget)
    tracker = Tracker(metric)
    
    #可以从设置参数中获得input，也可以从函数调用时直接传入
    writeVideo_flag = True
    #处理视频成一帧一帧
    video_capture = cv2.VideoCapture(input)


    if writeVideo_flag:
    # Define the codec and create VideoWriter object
        w = int(video_capture.get(3))
        h = int(video_capture.get(4))
        fourcc = cv2.VideoWriter_fourcc(*'MJPG')
        out = cv2.VideoWriter('./output/output.avi', fourcc, 15, (w, h))
        list_file = open('detection_rslt.txt', 'w')
        frame_index = -1

    fps = 0.0
    #循环处理每一帧的图片
    while True:
        #获取每一帧图片为三维列表的形式
        ret, frame = video_capture.read()  # frame shape 640*480*3
        if ret != True:
            break
        t1 = time.time()
        image = Image.fromarray(frame[..., ::-1])  # bgr to rgb
        #将获取的每一帧图片进行YOLO识别检测并画出框框，并使用deepsort对目标对象进行追踪
        boxs, confidence, class_names,frame,record= yolo.detect_image(image)
        frame = np.array(frame)
        # RGBtoBGR满足opencv显示格式
        frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)

        fps = (fps + (1. / (time.time() - t1))) / 100
        print("fps= %.2f" % (fps))
        #处理在图片上显示帧数
        frame = cv2.putText(frame, "fps= %.2f" % (fps), (0, 40), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)

        # cv2.imshow("video", frame)
        c = cv2.waitKey(1) & 0xff
        if c == 27:
            capture.release()
            break
        features = encoder(frame,boxs)
        # score to 1.0 here).
        detections = [Detection(bbox, 1.0, feature) for bbox, feature in zip(boxs, features)]
        # Run non-maxima suppression.
        boxes = np.array([d.tlwh for d in detections])
        scores = np.array([d.confidence for d in detections])
        indices = preprocessing.non_max_suppression(boxes, nms_max_overlap, scores)
        detections = [detections[i] for i in indices]

        # Call the tracker
        tracker.predict()
        tracker.update(detections)

        i = int(0)
        indexIDs = []
        c = []
        boxes = []

        for det in detections:
            bbox = det.to_tlbr()
            cv2.rectangle(frame,(int(bbox[0]), int(bbox[1])), (int(bbox[2]), int(bbox[3])),(255,255,255), 2)
            #print(class_names)
            #print(class_names[p])

        for track in tracker.tracks:
            if not track.is_confirmed() or track.time_since_update > 1:
                continue
            #boxes.append([track[0], track[1], track[2], track[3]])
            indexIDs.append(int(track.track_id))
            counter.append(int(track.track_id))
            bbox = track.to_tlbr()
            color = [int(c) for c in COLORS[indexIDs[i] % len(COLORS)]]
            #print(frame_index)
            list_file.write(str(frame_index)+',')
            list_file.write(str(track.track_id)+',')
            cv2.rectangle(frame, (int(bbox[0]), int(bbox[1])), (int(bbox[2]), int(bbox[3])),(color), 3)
            b0 = str(bbox[0])#.split('.')[0] + '.' + str(bbox[0]).split('.')[0][:1]
            b1 = str(bbox[1])#.split('.')[0] + '.' + str(bbox[1]).split('.')[0][:1]
            b2 = str(bbox[2]-bbox[0])#.split('.')[0] + '.' + str(bbox[3]).split('.')[0][:1]
            b3 = str(bbox[3]-bbox[1])

            list_file.write(str(b0) + ','+str(b1) + ','+str(b2) + ','+str(b3))
            #print(str(track.track_id))
            list_file.write('\n')
            #list_file.write(str(track.track_id)+',')
            cv2.putText(frame,str(track.track_id),(int(bbox[0]), int(bbox[1] -50)),0, 5e-3 * 150, (color),2)
            if len(class_names) > 0:
               class_name = class_names[0]
               cv2.putText(frame, str(class_names[0]),(int(bbox[0]), int(bbox[1] -20)),0, 5e-3 * 150, (color),2)

            i += 1
            #bbox_center_point(x,y)
            center = (int(((bbox[0])+(bbox[2]))/2),int(((bbox[1])+(bbox[3]))/2))
            #track_id[center]

            pts[track.track_id].append(center)

            thickness = 5
            #center point
            cv2.circle(frame,  (center), 1, color, thickness)

			# draw motion path
            for j in range(1, len(pts[track.track_id])):
                if pts[track.track_id][j - 1] is None or pts[track.track_id][j] is None:
                   continue
                thickness = int(np.sqrt(64 / float(j + 1)) * 2)
                cv2.line(frame,(pts[track.track_id][j-1]), (pts[track.track_id][j]),(color),thickness)
                #cv2.putText(frame, str(class_names[j]),(int(bbox[0]), int(bbox[1] -20)),0, 5e-3 * 150, (255,255,255),2)


        count = len(set(counter))
        #将所有行人数和和当前行人数和帧数显示在图片之上
        cv2.putText(frame, "Total Pedestrian Counter: "+str(count),(int(20), int(120)),0, 5e-3 * 200, (0,255,0),2)
        cv2.putText(frame, "Current Pedestrian Counter: "+str(i),(int(20), int(80)),0, 5e-3 * 200, (0,255,0),2)
        cv2.putText(frame, "FPS: %f"%(fps),(int(20), int(40)),0, 5e-3 * 200, (0,255,0),3)
        # cv2.namedWindow("YOLO4_Deep_SORT", 0);
        # cv2.resizeWindow('YOLO4_Deep_SORT', 1024, 768);
        # cv2.imshow('YOLO4_Deep_SORT', frame)
        #将所有行人数和和当前行人数和帧数显示在前端面板之上
        self.lcdNumber.setProperty("intValue", count)
        self.lcdNumber_2.setProperty("intValue", record)
        #将处理后的图片显示在面板上
        img = QImage(frame, frame.shape[1], frame.shape[0], frame.shape[1]*3,QImage.Format_RGB888)
        pix = QPixmap.fromImage(img)
        pix = pix.scaled(691,501, Qt.KeepAspectRatio, Qt.SmoothTransformation)
        self.label_4.setPixmap(pix)

        # show = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        #
        # showImage = QImage(show.data, show.shape[1], show.shape[0], QImage.Format_RGB888)
        #
        # self.label_4.setPixmap(QPixmap.fromImage(showImage))

        if writeVideo_flag:
            # save a frame
            out.write(frame)
            frame_index = frame_index + 1


        fps  = ( fps + (1./(time.time()-t1)) ) / 2
        out.write(frame)
        frame_index = frame_index + 1

        # Press Q to stop!
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

    print(" ")
    print("[Finish]")
    end = time.time()
    video_capture.release()

The front-end part (the front-end part is not designed by myself)

[External link image transfer failed. The source site may have an anti-leech link mechanism. It is recommended to save the image and upload it directly (img-x8UhEIVN-1612192346039)(BEA9D989F2654AD9AA8517C5CAE87C20)]

For the time being, the interface needs more design

But from the file import after clicking Start, I know that the reality is realized on the panel.

    #在面板上设置一个按钮，并将按钮的点击事件和打开文件想连
    self.pushButton.clicked.connect(self.openfile)
     
    def openfile(self):
        #设置打开文件页面的初始化位置
        path = "G:\\软件杯"
        #设置打开文件页面，并返回带有文件路径的结果
        #(['G:/软件杯/deep_sort_yolov4_fix/test_video/cut.mp4'], 'All Files (*)')
        openfile_name = QFileDialog.getOpenFileNames(None, "请选择要添加的文件", path, "Text Files (*.xls);;All Files (*)")
        #获取返回结果中的路径字符串形式
        path_list = openfile_name[0]
        path_name = path_list[0]
        #调用函数，产生等待界面，进行目标检测并追踪并且返回到主面板上
        mainback(self,path_name,YOLO())

#Front-end connection problem:

As I have encountered many problems with the front-end and back-end connection, I have two basic ideas:

1. The first one is to output various data processed from the back-end and paste it on the front-end panel. I found that the returned result is a picture frame by frame and the detection result corresponding to each picture is recycled. It takes a lot of effort to transmit the output results round by round to the front-end. The so-called method of transferring the back-end results to the front-end display is the second method.

2. The second method is that I instantiate the styled panel and input it to the back end, and paste the result directly on the panel in real time at the back end to display the back end result on the front end. End connection

The connection function is as follows:

mainback(self,path_name,YOLO())

1) Parameters: YOLO()
was originally the main function of the back-end main.py. It is called directly by the main function, and a YOLO object needs to be passed in. So our mainback function still needs the parameter YOLO()

 if __name__ == '__main__':
     main(YOLO())

2) Parameters: self

We need to pass the main panel into the backend, so we need an object passed into the main panel to refer to the main panel class, and our mainback function is just in the button click trigger time, the second click offense function is located in the main panel In the class, so at this time self is a reference to the main panel, so it has all the attributes of this class, so we can directly pass self to the main panel object to the back end, and add the output we need to the panel object. End result

3) Parameters: path_name
G:\software cup\deep_sort_yolov4_fix\main.py
as the starting program of the backend. When calling directly, we need to enter the tracking category, video location and other information
as follows:

python main.py -c person -i test_vedio/test.avi

We can directly enter the above statement on the command line to run the program, and use the following statement on the front end to call the back end program (str=the above command line statement constitutes a string):

os.system(str)
subprocess.run(str)

This kind of call can only be realized to run the back-end program on the front-end, but it cannot realize the connection between the front-end and the back-end. If you want to make a data connection, you need to use the function call method. If only the video address information can be passed in, but how to pass in the panel and yolo The object will not be realized for the time being, so the function is used to achieve the same purpose. So we need to pass in the video we want to recognize, and the address of the video can be extracted from the result of our selection of the folder function, so we need to pass it to the backend in the form of function parameters.