dnn module in opencv

1.1. Introduction to DNN module

The deep learning module (DNN) in OpenCV only provides inference functions, does not involve model training, and supports a variety of deep learning frameworks, such as TensorFlow, Caffe, Torch and Darknet.

Why does OpenCV implement a deep learning module?

• Lightweight. The DNN module only implements the inference function, and the amount of code and compilation and operation overhead are much smaller than other deep learning model frameworks.

• Easy to use. The DNN module provides built-in CPU and GPU acceleration without relying on third-party libraries. If OpenCV was used in the project before, then the DNN module can easily add deep learning capabilities to the original project.

• Versatility. The DNN module supports a variety of network model formats, and users can directly use them without additional network model conversion. The supported network structure covers commonly used target classification, target detection and image segmentation categories, as shown in the following figure:

The DNN module supports multiple types of network layers, basically covering common network computing requirements.

2. Introduction to common methods

2.1.dnn.blobFromImage

blobFromImage(img, 
                  scalefactor=None, 
                  size=None, 
                  mean=None, 
                  swapRB=None, 
                  crop=None, 
                  ddepth=None):

parameter:

• image: image data read by cv2.imread
• scalefactor: scaling the pixel value, assuming scalefactor=1/255 and mean=None, it means that the pixel value is normalized to the interval [0,1], that is, img = img * scalefactor
• size: The size of the output blob (image), such as (netInWidth, netInHeight)
• mean: Subtract the mean value from each channel. Example: When mean=(10,20,30), img =( cv2.merge( img[: ,: ,0] -10,img[:, :, 1]-20, img[:, :, 2] -30)) * scalefactor
• swapRB: swap the first and last channel of a 3-channel picture, such as BGR-RGB
• crop: Whether to crop the image after resize. If crop=True, then, after the size of the input image is resized, one side corresponds to one dimension of size, and the value of the other side is greater than or equal to the other dimension of size; then from the resized image Crop in the center. If crop=Falseyou don’t need crop, just keep the aspect ratio of the picture
• ddepth: Depth of the output blob. Optional: CV_32F or CV_8U

2.2.dnn.NMSBoxes

Function: Perform NMS (non-maximum suppression) processing according to the given detection boxes and corresponding scores

NMSBoxes(bboxes, 
             scores, 
             score_threshold, 
             nms_threshold, 
             eta=None, 
             top_k=None)

parameter:

• boxes: bounding boxes to be processed
• scores: For the scores of the bounding box to be processed
• score_threshold: The score threshold used to filter boxes
• nms_threshold: threshold used by NMS
• indices: the index value of the bounding box retained after NMS processing
• eta: The correlation coefficient in the adaptive threshold formula:

• top_k: If top_k>0, keep at most top_k bounding box index values.

2.3. dnn.readNet

Role: Load the deep learning network and its model parameters