Recent concern the large model, tidy, memo.
1. ResNet, the original version caffe following structure:
InsightFace bit different Resnet realization, the first default will first convolution 7x7 3x3 replaced, and remove the pool operation (face recognition is smaller than the input resolution 112x112 ImageNet), also when the number of layers is greater than when using 101 first bottleneck again 3x3 1x1 configuration of another 1x1, resnet50 inside or two 3x3 convolution, such a simple look at it, the same layer resnet 50, Insightface previous version corresponds to a replacement 1x1 16 3x3 convolution, the parameter is the amount of change much. Look after ZhaoJ implemented pytorch version did not notice the difference, as follows:
2. ResNeXt, looks like the original version of lua, the same framework, to modify the internal bottleneck, cut paper map:
ResNet is to split into a plurality of groups convolution, as VGG that paper, stacked convolution module ResNet this simple addition to the design / implementation is simple and convenient, but also can reduce the problem of over-fitting to a particular set of data; and this complex manual design inception easier to achieve higher accuracy in a limited amount calculation, so the bottleneck is in ResNet 3x3 convolution to convolution packet to Concurrently both worlds.
3. DenseNet, characterized by the features in FIG ResNet summed into corresponding channel mosaic channel, which leads to more and about wide network, so each interposed between a DenseBlock TransitionBlock (1x1 convolution) do dimension reduction, torchvision there model , Insightface also has achieved:
Probably a long way to unfold:
4. DPN, Dual Path Network, Dual refers Resnet + Desenet, Insightface reference model description:
It will be appreciated that the structure ResNet to utilize similarities and DenseNet, so common to both the intermediate convolution of the 3 groups. Original packet will ResneXt in 3x3 convolution applied to them, buy Gains III. . .
5. SENet, in the original implementation here is caffe, so when paper directly combined in FIG Scale, may be a customized embedded in other network module unit, Squeeze and Exciation Block. I mainly want to make up for convolutional neural network is relatively small receptive fields, then will be too concerned about the problem of local information, Sequeeze part of each feature introduced nonlinear partial view of global pooling, Excitiation, the operation is to get right back to take the weight of each channel , just as the original structure of the painting, probably unfold like this:
See FIG thesis always a kind of illusion SE Block bottleneck and are connected in parallel, but it is not, no change in the structure of bottleneck, SE Block can be flexibly inserted in front of or behind it.