Deep Learning Basic CNN Series - Convolution Computing

Convolution calculation

Convolution is a method of integral transformation in mathematical analysis, and the discrete form of convolution is used in image processing. What needs to be explained here is that in the convolutional neural network, the implementation of the convolutional layer is actually a cross-correlation operation defined in mathematics , which is different from the definition of convolution in mathematical analysis. Here is the same as other The framework is consistent with the convolutional neural network tutorial, and both use cross-correlation operations as the definition of convolution. The specific calculation process is shown in the figure.

Cross-correlation calculation

Although the convolution layer gets its name from the convolution operation, we usually use the more intuitive cross-correlation operation in the convolution layer . In a two-dimensional convolutional layer, a two-dimensional input array and a two-dimensional kernel (kernel) array output a two-dimensional array through a cross-correlation operation. We use a concrete example to explain the meaning of the two-dimensional cross-correlation operation. As shown in FIG.

The convolution kernel (kernel) is also called a filter (filter) , assuming that the height and width of the convolution kernel are $k_h$and $k_w$, then it will be called $k_h\times k_w$Convolution, such as $3\times 5$ convolution means that the convolution kernel has a height of 3 and a width of 5.

---

• As shown in the above picture (a): the size of the picture on the left is $3\times 3$ , indicating that the input data is a dimension of$3\times A\; two-dimensional\; array\; of\; 3$ ; the size of the middle image is$2\times 2$ , means a dimension is$2\times A\; two-dimensional\; array\; of\; 2$ , we call this two-dimensional array a convolution kernel. First align the upper left corner of the convolution kernel with the upper left corner of the input data (ie: the (0,0) position of the input data), and multiply each element of the convolution kernel by the element in the input data whose position corresponds to it , and then add the convolution to get the first result of the convolution output:

                  $0\times 1+1\times 2+2\times 4+3\times 5=25$  (a)

The calculation methods of (b), (c) and (d) in the figure are the same as above, I believe that smart people don’t need me to demonstrate more.

The calculation process of the convolution kernel can be expressed by the following mathematical formula, where $a$ represents the input image,$b$ represents the output feature map,$w$ is the convolution kernel parameter, they are all two-dimensional arrays.

                      $b[i,j]=\sum _{u,v}a[i+u,j+v]\cdot w[u,v]$

For example, the size of the convolution kernel in the above figure is $2\times 2$，则$u$ can take 0 and 1,$v$ can also take 0 and 1, that is to say:

  $b[i,j]=a[i+0,j+0]\cdot w[0,0]+a[i+0,j+1]\cdot w[0,1]+a[i+1,j+0]\cdot w[1,0]+a[i+1,j+1]\cdot w[1,1]$

We can verify its correctness, when $[i,j]$ take different values, whether the result calculated according to this formula is consistent with the example in the above figure.

Supplement:
In a convolutional neural network, in addition to the convolution process described above, a convolution operator also includes the operation of adding a bias term. For example, assuming that the bias is 1, the result of the above convolution calculation is:
$$0\times 1+1\times 2+2\times 4+3\times 5+1=26$$

$$0\times 2+1\times 3+2\times 5+3\times 6+1=32$$

$$0\times 4+1\times 5+2\times 7+3\times 8+1=44$$

$$0\times 5+1\times 6+2\times 8+3\times 9+1=50$$

practise

After learning the knowledge, let's do a question for the next exercise to help us fully grasp the operation of convolution.
Title: Calculate how many multiplication and addition operations there are in the convolution. The
input data shape is $[10,3,224,224]$ , convolution kernel$k_h=k_w=3$ , the number of output channels is 64,$stride=1$ , fill$p_h=p_w=1$ .
Then to complete such a convolution, how many multiplication and addition operations do you need to do in total?

• Tips
  First look at how many multiplication and addition operations need to be done to output a pixel, and then calculate the total number of operations required.

Question steps:

1. First consider the two-dimensional convolution when there is only one input channel:
   Assuming that the output is B and the input is A, first calculate a pixel of B,
   
   among which there are a total of 9 multiplications and 8 additions.
   But generally the pictures we input are RGB three-channel, so we need to calculate each channel $B_{00}^{(c=0)},B_{00}^{(c=1)},B_{00}^{(c=2)}$, the total number of multiplication operations is $3\times 9=27$ , the number of addition operations is$3\times 8=24$ times.

2. Then add the values ​​​​of these input channels, and add the bias parameter $b$
   $$B_{00}=B_{00}^{(c=0)}+B_{00}^{(c=1)}+B_{00}^{(c=2)}+b$$
   Since 3 additional addition operations need to be introduced, the final total number of addition operations is$24+3=27$
   From this, it can be obtained that the number of multiplication operations required to calculate a pixel point is 27, and the number of addition operations is also 27.

3. The size of the output feature map is $[10,64,224,224]$ , the total number of multiplication operations required is:
   $$27\times 10\times 64\times 224\times 224=867041280$$
   The number of addition operations is the same as the number of multiplication operations is 867041280.