Depth separable convolution (calculation process and code implementation)

1. Basic introduction

        Depth-separable convolution is an improvement over traditional convolution, which reduces the number of parameters required for convolution calculations by splitting the correlation between spatial and channel dimensions. The depthwise separable convolution calculation mainly includes two parts. The first part is Depthwise Convolution, which splits the channels and convolves each channel with a separate convolution kernel. That is, the number of convolution kernels is equal to the number of channels of the input features, and each convolution kernel has only one channel. The second part is Pointwise Convolution, which is a 1×1 convolution, and the number of convolution kernel channels is the same as the number of input channels.

2. Traditional convolution calculation process

        In the traditional convolution process, the number of channels of the convolution kernel needs to be the same as the number of input channels. The operation of each convolution kernel includes two parts. The first part is the convolution operation between the corresponding channel of the convolution kernel and the corresponding channel of the input. (Corresponding positions are multiplied and then added), the second part is to add the convolution results of different channels to obtain a feature map with a channel number of 1. n convolution kernels finally obtain an output with n channels.

2. Depth separable convolution calculation process

 

        Depth separable convolution is divided into two modules. The first part is depthwise convolution. Unlike the traditional convolution process, depthwise convolution separates channels, and the number of convolution kernels is the same as the number of channels of input features. Each convolution The number of channels of the kernel is 1, and a convolution kernel only operates on one channel. The final number of feature channels generated is the same as the number of input channels.

The second part is pointwise convolution, which has two main functions:

1. Freely set the number of output channels. The number of output channels is equal to the number of one-dimensional convolution kernels in pointwise convolution.

2. Fusion of information between channels, depthwise convolution ignores information between channels, and information between channels can be fused through one-dimensional convolution.

3. Depth separable convolution implementation

Implement depthwise separable convolutions:

import torch
from torch import nn
from torchsummary import summary
class depth_separable(nn.Module):
    def __init__(self, in_channels:int, out_channels:int) -> None:
        super(depth_separable,self).__init__()
        self.depth_conv = nn.Conv2d(   #和常规卷积不同就是设置了groups参数
        in_channels,
        in_channels,
        kernel_size=3,
        stride=1,
        groups=in_channels,      #groups设置为输入通道数，可以使逐通道卷积
        )
        self.point_conv = nn.Conv2d(   #实现点卷积
        in_channels,
        out_channels,
        kernel_size=1,
        )
    def forward(self, x):

        return self.point_conv(self.depth_conv(x))
    
class mydepth_separable(nn.Module):
    def __init__(self) -> None:
        super(mydepth_separable,self).__init__()
        self.conv2d = depth_separable(3,8)
        self.relu = nn.ReLU()
    def forward(self, x):
        
        return self.relu(self.conv2d(x))
device = torch.device("cuda" )
model=mydepth_separable().to(device)
summary(model, (3, 5, 5))     #查看参数量（3,5,5）表示输入的尺寸是5×5×3

 Parameter result:

 It can be seen that the output of depthwise convolution is 3×3×3, the parameter amount is 30, the output of point convolution is 3×3×8, and the parameter amount is 32. The parameter amount here includes the bias.

Ordinary convolution:

import torch
from torch import nn
from torchsummary import summary
class conv(nn.Module):
    def __init__(self, in_channels:int, out_channels:int) -> None:
        super(conv,self).__init__()
        self.conv = nn.Conv2d(   
        in_channels,
        out_channels,
        kernel_size=3,
        stride=1,
        )
    def forward(self,x):
        return self.conv(x)
class myconv(nn.Module):
    def __init__(self) -> None:
        super(myconv,self).__init__()
        self.conv2d=conv(3,8)
        self.relu=nn.ReLU()
    def forward(self,x):
        return self.relu(self.conv2d(x))
device = torch.device("cuda" )
model1=myconv().to(device)
summary(model1, (3, 5, 5))  

         Compared with ordinary convolution and depth-separable convolution, depth-separable convolution requires only 62 parameters, while ordinary convolution requires 224 parameters. This is the role of depth-separable modules, which greatly reduces the amount of parameters. However, when the number of parameters is reduced, the performance of the model may also decrease. Therefore, how to reduce the network size while ensuring the performance of the network is also a research hotspot for lightweight networks.