DNN module in OpenCV

Introduction to DNN module: 

        Since version 3.3, OpenCV has added support for deep learning networks, namely the DNN module, which supports the generation of mainstream deep learning frameworks and the loading of models everywhere.

        The deep learning module (DNN) in OpenCV only provides reasoning functions, does not involve model training, and supports a variety of deep learning frameworks, such as TensorFlow, Caffe, Torch and Darknet.

• lightweight. The DNN module only implements the reasoning function, and the amount of code and compilation and running overhead are much smaller than other deep learning model frameworks.

• Easy to use. The DNN module provides built-in CPU and GPU acceleration without relying on third-party libraries. If OpenCV was used in the project before, it is very convenient to add deep learning capabilities to the original project through the DNN module.

• Versatility. The DNN module supports a variety of network model formats, and users can use it directly without additional network model conversion. The supported network structures cover the commonly used categories of object classification, object detection, and image segmentation

 一、dnn.blobFromImage

Function: According to the input image, create blobs in the order of dimension N (number of pictures), number of channels C, height H and width W.

blobFromImage(image, 
              scalefactor=None, 
              size=None, 
              mean=None, 
              swapRB=None, 
              crop=None, 
              ddepth=None):

parameter:

• image: The image data read by cv2.imread

• scalefactor: Scale pixel values, such as [0, 255] - [0, 1]

• size: The size of the output blob (image), such as (netInWidth, netInHeight)

• mean: Subtract the mean value from each channel. If the input image is in BGR order and swapRB=True, the channel order is (mean-R, mean-G, mean-B).

• swapRB: Swap the first and last channel of a 3-channel image, such as BGR - RGB

• crop: Whether to crop after the image size is resized. If crop=True, then, after the input image is resized, one side corresponds to one dimension of size, and the value of the other side is greater than or equal to another dimension of size; then from the resized image Crop in the center. If crop=Falseso, no crop is needed, just keep the aspect ratio of the image

• ddepth: Depth of output blob. Optional: CV_32F or CV_8U

Example:

import cv2
from cv2 import dnn
import numpy as np 
import matplotlib.pyplot as plt

img_cv2 = cv2.imread("test.jpeg")
print("原图像大小: ", img_cv2.shape)

inWidth = 256
inHeight = 256
outBlob1 = cv2.dnn.blobFromImage(img_cv2,
                                scalefactor=1.0 / 255,
                                size=(inWidth, inHeight),
                                mean=(0, 0, 0),
                                swapRB=False,
                                crop=False)
print("未裁剪输出: ", outBlob1.shape)
outimg1 = np.transpose(outBlob1[0], (1, 2, 0))

outBlob2 = cv2.dnn.blobFromImage(img_cv2,
                                scalefactor=1.0 / 255,
                                size=(inWidth, inHeight),
                                mean=(0, 0, 0),
                                swapRB=False,
                                crop=True)
print("裁剪输出: ", outBlob2.shape)
outimg2 = np.transpose(outBlob2[0], (1, 2, 0))

plt.figure(figsize=[10, 10])
plt.subplot(1, 3, 1)
plt.title('输入图像', fontsize=16)
plt.imshow(cv2.cvtColor(img_cv2, cv2.COLOR_BGR2RGB))
plt.axis("off")
plt.subplot(1, 3, 2)
plt.title('输出图像 - 未裁剪', fontsize=16)
plt.imshow(cv2.cvtColor(outimg1, cv2.COLOR_BGR2RGB))
plt.axis("off")
plt.subplot(1, 3, 3)
plt.title('输出图像 - 裁剪', fontsize=16)
plt.imshow(cv2.cvtColor(outimg2, cv2.COLOR_BGR2RGB))
plt.axis("off")
plt.show()

 二、dnn.NMSBoxes

Function: Perform NMS (non-maximum suppression) processing according to the given detection boxes and corresponding scores.

NMSBoxes(bboxes, 
         scores, 
         score_threshold, 
         nms_threshold, 
         eta=None, 
         top_k=None)

parameter:

• boxes: bounding boxes to be processed

• scores: scores for the pending bounding box

• score_threshold: score threshold for filtering boxes

• nms_threshold: The threshold used by NMS

• indices: the index value of the bounding box retained after NMS processing

• eta: correlation coefficient in the adaptive threshold formula

• top_k: If top_k>0, keep at most top_k bounding box index values.

3. dnn.readNet

Role: Load the deep learning network and its model parameters

readNet(model, config=None, framework=None)

parameter:

• model: The model binary file of the trained weight parameters, the supported formats are: *.caffemodel(Caffe), *.pb(TensorFlow), *.t7 or  *.net(Torch),  *.weights(Darknet), *.bin(DLDT).

• config: A text file containing network configuration, supported formats are: *.prototxt (Caffe), *.pbtxt(TensorFlow), *.cfg (Darknet), *.xml (DLDT).

• framework: the framework name of the supported format

1. Load the configuration network and training weight parameters using Caffe

readNetFromCaffe(prototxt, caffeModel=None)

2. Load the configuration network and training weight parameters using Darknet

readNetFromDarknet(cfgFile, darknetModel=None)

3. Load the configuration network and training weight parameters using Tensorflow

readNetFromTensorflow(model, config=None)

4. Load the configuration network and training weight parameters using Torch

readNetFromTorch(model, isBinary=None)

5. Load the .onnx model network configuration parameters and weight parameters

readNetFromONNX(onnxFile)