CNN Network Structure3

• Foreword :…
  
  
• 
  
  

0 More Details: HomeLink

1 MobileNet-V1

1.1 Depthwise Conv

• Traditional convolution operation VS the convolution operation mentioned in the paper.
  This part is still quite important. The calculation amount of Depthwise Convolution is much smaller than Traditional Convoulution.
  
  
• For example, take a simple example
  
  
• Found it, under normal circumstances, compared to the standard convolution calculation, the calculation amount is reduced by 8 to 9 times, but there is only 1% accuracy loss.

• In general, Structure-Macro looks like this:
  How to draw it? Use the caffe network file .prototx, copy directly to the netscope, draw the caffe model online, after the copy is completed, shift+enter can see the network structure Up.
  
  

1.2 Hyperparameters

• The above two points strongly felt that they were of little significance, and even thought they were used for thesis.

1.3 Training-Some Details

• RMSprop I didn’t read this, I don’t understand
  Less regularization and data augmentation considering small models have less trouble overfitting The data set is not too complicated
  Very Little or no weight decay (l2 regularization) on depthwise filters since there are so few parameters in them.
  Here is an example , MobileNet + SSD in
  
  short, the important thing is 1.1 there

2 MobileNet-V2

2.1 Inverted Residuals & Linear Bottlenecks

The main contribution of the paper-high forced grid version:
Inverted Residuals + Linear Bottlenecks
Mandarin version: definitely increase the number of channels in the middle of each layer to increase the number of parameters, and then retract before the next layer...


First impression: very little parameters, very slim Blobs
mental torture: Then how could it be better?


Original residual: reduce-transffer-expand
Original bottlenecks: Eltwise + with ReLU at the end of a bottleneck
Inverted residual: expand-transfer – reduce
Linear Bottlenecks: Eltwise+ with NO ReLU at the end of a bottleneck


quality explanation:
Inverted Residuals: The function is to uncompress the data, so that the low-dimensional manifold of interest we are interested in can be included in our high-dimensional space.
Linear Bottlenecks: Neural networks are believed to be able to embed MOI in low-dimensional spaces. If the MOI of the current activation space is relatively complete, ReLU will collapse the space and lose information; and its non-zero part is a linear transformation, which is actually a linear classifier. Therefore, the linear bottleneck
Manifold is used: Manifold-a space with local European spatial properties-


Mandarin interpretation:
Inverted Residuals: Skip connection This bottleneck structure has proved to be very effective, so I want to use it; but if the channel is compressed first as before, the number of channels is already small and then compressed, so it is better to increase and then decrease.
Linear Bottlenecks: ReLU makes the negative half axis 0. Originally, I didn't have many parameters, and my learning ability was limited. Now that some parameters are set to 0, I can't learn anything. Just don't ReLU in eltwise+.


Structure: Stride = 1 block is different from Stride = 2 block, eh.

2.2 Effect

In other words, V2 is better, faster and has fewer parameters

3 Summary

V1与V2如何加速的总结额
MobileNet-V1-Accelerate conv operation:
Mainly separate traditional conv into 2 steps: Depthwise + Pointwise
MobileNet-V2-Add more useful parameters:
Inverted residual: expand – transfer – reduce
+
Linear bottleneck: remove ReLU after Eltwise+