SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Paper address: Arxiv Paper
Github: Caffe
design concept:
-
Replace 3x3 conv with 1x1 conv
The parameter amount of 1x1conv is 1/9 of 3x3conv
Replace 3x3 with partial conv1x1 in expand, in order not to affect Accuracy
- Reduce ic (input channel) of conv3x3
Reduce the dimensionality of the input of expand through conv1x1 in squeeze, that is, reduce the ic corresponding to expand
- Delay downsampling
The previous layers have larger feature maps, which is beneficial to improve the Accuracy of the model
Downsampling method: strides>1 convolutional layer, pooling layer
Fire Module
Modular convolution key points:
- squeeze conv1x1 layer: Use conv1x1 for channel dimensionality reduction, reaching the first point in 1
- Expand conv1x1+conv3x3 layer: Partially use conv1x1 instead of conv3x3,
-
Adjustable parameters
e1x1 (expand convolution layer中conv1x1的output channel)
-
- The fire module is actually a deformation of the bottle neck module that is not much different from the InceptionV1 module, except that several scales of convolution are less done.
Network structure
- Left: Standard squeezeNet
- Middle: Added residual bypass structure
- Right: Added complex bypass structure
Detailed description of parameters
- Parameter calculation formula: oc x ic x kh x kw
-
Original parameter calculation without Fires module
input 55x55x96, output 55x55x128
Parameter amount: 128x96x3x3 = 110,592 ( I don't understand why it is 11920 in Table1? )
-
Add parameter calculation of Fire2 module
Squeeze conv1x1: 96x16x1x1
Expand conv1x1: 16x64x1x1
Expand conv3x3: 16x64x3x3x 1/3(sparsity)
Amount of parameters: 96x16x1x1 + 16x64x1x1 + 16x64x3x3x 1/3 = 4,096 ( not 5746 in Table1? )
Reference