(3) Grayscale image processing, judging whether the previous frame image is empty. If it is empty, assign a frame of image and perform image grayscale processing; if not empty, also perform image grayscale processing.

(4) Frame difference method, using the previous frame image and this frame image for difference processing.

(5) Binarization image processing, which performs binarization processing on the image after the frame difference method.

(6) Median filter processing

(7) Closed operation, erosion, expansion processing.

(8) Perform contour detection on the moving image and return the coordinate set

(9) Use the coordinates to frame the target.

(10) Display the image, and change the processed image to the previous frame image.

2.2 Parameters and meanings of opencv call function

2.2.1 ret, img = cap.read() read frame image

cap is the camera image data captured by your camera device 0 (eg: cap = cv2.VideoCapture(0) );

read() is the opencv library function to read a frame of image from the camera, and cap.read() is to obtain a frame of image read by camera device 0;

img is a frame of image read by camera device 0, and the image format is BGR;

ret is bool type, it returns True if the read is successful, and False if it fails.

2.2.2 cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) grayscale image

img: The image to be converted.

cv2.COLOR_BGR2GRAY: Convert a picture in BGR format to a grayscale picture.

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) gray is the converted grayscale image

2.2.3 gray_diff_img = cv2.absdiff(gray_img, previous_img) 帧差法

cv2.absdiff(a,b) : a and b are both a frame of image.

Compare the difference between the two frames of images (for example: during motion detection, only the moving object will move, but the background will not move, cv2.absdiff(a,b) will get the picture of the moving object with the background removed ) . gray_diff_img is a frame of image after two frames of processing.

2.2.4 cv2.threshold (src, thresh, maxval, type) binarization processing

src: A frame of image passed in.

thresh: threshold (value range 0-255, selected according to light intensity)

maxval: fill color (value range 0-255, generally choose 255 for binarized images)

type: as shown below

thresh_img = cv2.threshold(gray_diff_img, 40, 255, cv2.THRESH_BINARY)

The resulting binarized image thresh_img

2.2.5 cv2.medianBlur(src, ksize) median filter

src is the image to process.

ksize is the size of the filter kernel. The filter kernel size refers to the height and width of its neighborhood image during the filtering process. It should be noted that the core size must be an odd number greater than 1, such as 3, 5, 7, etc.
Example: mask_img = cv2.medianBlur(thresh_img, 3)

2.2.6 np.ones(shape, dtype=None, order='C')

shape: is an int or a tuple of ints that define the size of the array.

dtype: is an optional parameter, the default value is float. It is used to specify the data type of the array, such as int.

order: The order defines whether to store multidimensional arrays in memory in row-major (C-style) or column-major (Fortran-style) order. We don't need to write, just default.

Example: k = np.ones((3, 3), np.uint8)

A small matrix of (1,1) with a size of 1 is a unit to form a large matrix with 3 rows and 3 columns, and the type is uint8. (I don’t know exactly why it is used, I guess it probably means the size of the image)

2.2.7 cv2.morphologyEx(src, op, kernel)

src: is the image that needs to be processed.

op：

cv2.MORPH_OPEN: open operation, corrode and then expand the image, which is equivalent to dilate(erode(src,kernal)), open operation smoothes the boundary of the image, removes bumps, etc. cv2.MORPH_CLOSE: close operation, first executes the
image Perform expansion and corrosion, which is equivalent to erode(dilate(src,kernal)), and the closing operation is used to fill small holes inside the image, fill the depression of the image, etc.

kernel: Input an array as the kernel, representing the image size. (I don't really understand either)

例 close = cv2.morphologyEx(mask_img, cv2.MORPH_CLOSE, k)

2.2.8 cv2.findContours(image,mode,method)

image : Represents the input image. Note that the input image must be a binary image. If the input image is a color image, it must be grayscaled and binarized first.

mode : Indicates the retrieval mode of the outline, there are 4 types:

cv2.RETR_EXTERNAL means only detect the outer contour.

Contours detected by cv2.RETR_LIST do not establish a hierarchical relationship.

cv2.RETR_CCOMP establishes two levels of contours, the upper layer is the outer boundary, and the inner layer is the boundary information of the inner hole. If there is another connected object in the inner hole, the boundary of this object is also on the top layer.

cv2.RETR_TREE builds the outline of a hierarchical tree structure.

method : Approximation method for the contour, there are 4 types:
cv2.CHAIN_APPROX_NONE stores all the contour points, and the pixel position difference between two adjacent points does not exceed 1, namely max(abs(x1-x2), abs(y2-y1 ))<=1.
cv2.CHAIN_APPROX_SIMPLE compresses elements in the horizontal direction, vertical direction, and diagonal direction, and only retains the end coordinates of the direction. For example, a rectangular contour only needs 4 points to save the contour information.
cv2.CHAIN_APPROX_TC89_L1 and cv2.CHAIN_APPROX_TC89_KCOS use teh-Chinl chain approximation algorithm.

Function return value contours : Returns several contours (returns several moving object frames), and the unit of each contour is a point set (that is, the contour size)

例：cnts = cv2.findContours(close_img,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_NONE)

2.2.9 cv2.contourArea() Calculate the contour area

2.2.10 cv2.boundingRect() returns the coordinates of the outline frame

x, y, w, h = cv2.boundingRect(c) , x, y, w, h are the coordinate length of the contour, c is a subset of contours

2.2.11 cv2.rectangle() Coordinate connection, frame outline

cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 2)

Image, coordinates of the upper left corner of the rectangle, coordinates of the lower right corner of the rectangle, font color, font weight

2.2.12 cv2.imshow() window display image

cv2.imshow("name", img) "name" is the window name, and img is the window display image.

3. Processing code

import cv2
import time
import numpy as np

def videos():
    cap = cv2.VideoCapture(0)
    cap.set(3, 640)
    cap.set(4, 480)
    cap.set(5,40)
    
    img_num = 0
    k = np.ones((3, 3), np.uint8)  

    while True:
        ret, img = cap.read()

        if not img_num:
            previous = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

        gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
        gray_diff = cv2.absdiff(gray, previous)  
        thresh = cv2.threshold(gray_diff, 40, 255, cv2.THRESH_BINARY)[1] 
        mask = cv2.medianBlur(thresh, 3) 
        close = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, k) 

        cnts = cv2.findContours(close,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_NONE)[0] 
        for c in cnts:   
            area = cv2.contourArea(c)
            if area > 200:
                x, y, w, h = cv2.boundingRect(c)
                cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 2)

        cv2.imshow("thresh_img", close)
        cv2.imshow("Result", img)
        img_num += 1

        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

videos()

The upper left corner is a little white due to sunlight.

Raspberry Pi CSI camera uses python to call opencv library function for motion detection and recognition

1. Complete the call of the camera

2. Use python to call the opencv library function to process the image

2.1 General flow of image processing