1. Background

Taking the two-level cascade model as an example, the first-level target detection model is used to detect people, and the second-level target detection model is used to detect mobile phones, walkie-talkies, etc. Then, during the actual data collection process, devices such as mobile phones and walkie-talkies are not in the first-level detection frame of personnel, so that there are fewer samples for the second-level model training .

In the training phase of the secondary target detection model, when sampling data, we should not only use the cropped image of the primary target detection frame as input, but also need to dynamically crop the original image to include devices such as mobile phones and walkie-talkies to reduce background interference.
input original image
First-level target frame area ROI area of interest
The above three images are the original image, the first-level target detection frame area, and the desired target area during training (other mobile phones need to be included).

Basic idea: Based on the mouse event of opencv, draw a rectangular frame on the original picture to get the area of interest and save it.

2. OpenCV mouse event

2.1 Set mouse events

setMouseCallback(const String & 	winname,  #要设置鼠标事件的窗口名
				 MouseCallback 	onMouse,      #回调函数，当指定窗口产生鼠标事件时，调用的函数名
				 void* userdata = 0)          #传递给回调函数的可选参数

2.2 Callback function prototype

void MouseCallback(int event,        #鼠标基础事件
             int x, int y,           #鼠标在x轴y轴方向上的坐标值，窗口左上角为原点(0,0)
             int flags,              #flags的值代表鼠标拖拽事件和Ctrl、Shift、Alt按键事件的代号
             void *userdata)         #用户数据

2.3 Mouse Events

2.3.1 event basic event

This corresponds to the first parameter event of the calling function. When the mouse performs the corresponding action, the event will become the corresponding value.

enum
{
    
    
    CV_EVENT_MOUSEMOVE      =0,   //鼠标移动
    CV_EVENT_LBUTTONDOWN    =1,   //按下左键
    CV_EVENT_RBUTTONDOWN    =2,   //按下右键
    CV_EVENT_MBUTTONDOWN    =3,   //按下中键
    CV_EVENT_LBUTTONUP      =4,   //放开左键
    CV_EVENT_RBUTTONUP      =5,   //放开右键
    CV_EVENT_MBUTTONUP      =6,   //放开中键
    CV_EVENT_LBUTTONDBLCLK  =7,   //左键双击
    CV_EVENT_RBUTTONDBLCLK  =8,   //右键双击
    CV_EVENT_MBUTTONDBLCLK  =9,   //中键双击
    CV_EVENT_MOUSEWHEEL     =10,  //滚轮滚动
    CV_EVENT_MOUSEHWHEEL    =11   //横向滚轮滚动
};

2.3.2 flag drag event

This corresponds to the fourth parameter of the callback function. When there is a mouse drag or a combination of CTRL, shift, and ALT, the flag becomes the corresponding value.

 
enum
{
    
    
    CV_EVENT_FLAG_LBUTTON   =1,   //左键拖拽
    CV_EVENT_FLAG_RBUTTON   =2,   //右键拖拽
    CV_EVENT_FLAG_MBUTTON   =4,   //中键拖拽
    CV_EVENT_FLAG_CTRLKEY   =8,   //按住CTRL拖拽
    CV_EVENT_FLAG_SHIFTKEY  =16,  //按住Shift拖拽
    CV_EVENT_FLAG_ALTKEY    =32   //按住ALT拖拽
};

在这里插入代码片

3. Dynamic cropping of images

The specific implementation code is as follows:

import cv2
import os

ROOT = "data7"		# 原图路径

img_cnt = 0

def mouse_callback(event, x, y, flags, param):
    global img_data, point1, point2,g_rect, img_cnt
    img2 = img_data.copy()
    if event == cv2.EVENT_LBUTTONDOWN:  # 左键点击,则在原图打点
        print("1-EVENT_LBUTTONDOWN")
        point1 = (x, y)
        cv2.circle(img2, point1, 10, (0, 255, 0), 5)
        cv2.imshow('image', img2)
 
    elif event == cv2.EVENT_MOUSEMOVE and (flags & cv2.EVENT_FLAG_LBUTTON):  # 按住左键拖曳，画框
        print("2-EVENT_FLAG_LBUTTON")
        cv2.rectangle(img2, point1, (x, y), (255, 0, 0), thickness=2)
        cv2.imshow('image', img2)
 
    elif event == cv2.EVENT_LBUTTONUP:  # 左键释放，显示
        print("3-EVENT_LBUTTONUP")
        point2 = (x, y)
        cv2.rectangle(img2, point1, point2, (0, 0, 255), thickness=2)
        cv2.imshow('image', img2)
        if point1!=point2:
            min_x = int(min(point1[0], point2[0]))
            min_y = int(min(point1[1], point2[1]))
            width = int(abs(point1[0] - point2[0]))
            height = int(abs(point1[1] - point2[1]))
            g_rect=[min_x,min_y,width,height]
            cut_img = img_data[min_y:min_y + height, min_x:min_x + width]   # 从原图上裁剪
            cv2.imwrite("crop_img_{}.jpg".format(img_cnt), cut_img)         # 图像文件保存
            print("save image.")
            img_cnt += 1


if __name__ == "__main__":
    img_lists = os.listdir(ROOT)

    cv2.namedWindow("image")
    cv2.setMouseCallback("image", mouse_callback)


    for img in img_lists:
        img_path = os.path.join(ROOT, img)
        if os.path.exists(img_path):
            print("file existed.")
        img_data = cv2.imread(img_path)

        cv2.imshow("image", img_data)
        k = cv2.waitKey(0)

        if k == ord('q'):
            break

cv2.destroyAllWindows()

The general steps are as follows:

Create a new window and set the callback function;
Read and display raw image data;
Capture the mouse events of left-click, left-click drag, and left-click release, and draw the area target frame;
And save the area target;

4. Summary

This article mainly introduces a solution for the case of two-level model cascading when the training samples of the second-level target detection algorithm are small. Through the above script, the artificial clipping area can be visualized and the training samples can be saved to reduce background interference.

Image dynamic cropping