Group convolution in torch

Ordinary convolution

If the channel of the input feature is 32 , and the channel of the output feature is expected to be 64 , for the ordinary convolution of 3*3 , the convolution layer is set to:

conv = torch.nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3)

We look at the parameter volume of the convolution kernel

print(conv.weight.shape) # torch.size([64,32,3,3])

So the parameter amount of ordinary convolution is 64 $\times$ 32 $\times$ 3 $\times$ 3 = 18432 parameters

group convolution

The channel of the same input feature is 32 , and the channel of the expected output feature is 64. If group convolution is used , the convolution layer is set to:

conv = torch.nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3, groups=8)

We look at the parameter volume of the convolution kernel

print(conv.weight.shape) # torch.size([64,4,3,3]) 这里 4 表示 in_channels/groups = 4

So the parameter amount of group convolution is 64 $\times$ 4 $\times$ 3 $\times$ 3 = 2304 parameters, the same output channel, the parameters are reduced by 8 times

In order to achieve the same output effect, the method of group convolution is:

• Divide 32 input channels into 8 groups of 4 channels
• put 64 $\times$ 4$\times$ 3$\times$ 3 convolution kernels are divided into 8 groups, each group of parameters is 8$\times$ 4 $\times$ 3 $\times$ 3
• In this way, 8 sets of inputs and 8 sets of convolution kernels are one-to-one for ordinary convolution, and 8 sets of output features with 8 output channels can be obtained. By splicing the 8 sets of output results together, the output results of 64 channels can be obtained.