When talking about MobileNet, MobileNetV2 and MobileNetV3, we are referring to a series of lightweight deep learning-based neural network architectures designed to reduce the computational and parameter size of the model while maintaining a high degree of accuracy. The following is a detailed summary of their respective innovations:
- MobileNet：
- Depth separable convolution : MobileNet introduced depth separable convolution for the first time, dividing standard convolution into two stages: depth convolution and point-wise convolution, thus significantly reducing the amount of calculation. This convolution method effectively reduces the number of parameters and computational complexity of the model, making it possible to run the model on embedded devices and mobile devices.
- Width multiplier and resolution multiplier : MobileNet introduces the concepts of width multiplier and resolution multiplier, making it possible to adjust the size and complexity of the model as needed. The model size can be further reduced by reducing the number of channels and input resolution.
- MobileNetV2：
- Linear bottleneck structure : MobileNetV2 improves the foundation of the original MobileNet and introduces a linear bottleneck structure. This structure applies traditional nonlinear activation functions before convolution, reducing information loss while still maintaining lightweight characteristics.
- Inverted residual structure and dilated convolution : MobileNetV2 introduces the inverted residual structure, which first performs dilated convolution (Dilated Convolution) and then performs fusion convolution, thus improving the nonlinear representation ability of the network. This structure improves the accuracy of the model while maintaining computational efficiency.
- Inverted Residuals with Linear Bottlenecks : MobileNetV2 adopts an "inverted residual" structure, which combines traditional residual blocks with linear bottlenecks to further improve feature representation capabilities.
- MobileNetV3：
- Network Architecture Search : MobileNetV3 introduces Network Architecture Search (NAS) to find better network structures. By automatically searching and selecting the appropriate block type, number of channels, and resolution, MobileNetV3 further reduces the model's computational load while maintaining high accuracy.
- H-Swish activation function : MobileNetV3 introduces a new activation function-Hard-Swish (H-Swish). Compared with the traditional ReLU activation function, H-Swish provides smoother nonlinear characteristics while maintaining efficient calculations, helping to improve the accuracy of the model.
- Adjustable attention module : MobileNetV3 introduces an adjustable attention module so that the network can adaptively learn the importance of features. This helps further improve the performance of the model.
In general, the innovation points of MobileNet, MobileNetV2 and MobileNetV3 mainly focus on the improvement of convolution structure, optimization of network architecture, innovation of activation function and the introduction of attention mechanism. These innovations enable these lightweight models to achieve excellent performance on mobile devices and embedded systems while maintaining high levels of accuracy even with limited computing resources.