Depth range of neural networks, and how their performance?
Practical applications, under various constraints speed, memory, accuracy, etc., which models should try as a backbone?
There are paper each network model are compared and analyzed, to form a complete perspective of a view of all the major models, the analysis results can provide guidance and assistance in practice.
This blog is mainly incorporates the conclusions of the article 3, respectively
- 201605-An Analysis of Deep Neural Network Models for Practical Applications
- 201809-Analysis of deep neural networks
- 201810-Benchmark Analysis of Representative Deep Neural Network Architectures
Articles 1 and 3 are paper, 2 is a blog post (update 1). This article three of the major state image recognition tasks (ImageNet-1k) a comparative analysis of the art network, indicators used are:
- Accuracy , accuracy, only cental crop, evaluation Top1, Top5 accuracy on the ImageNet-1k
- Complexity Model , complexity of the model, by the amount of learning the model parameters can be measured (approximately model file size), reflecting the degree of freedom
- Complexity Computational , computational complexity, the number of operations by the floating-point operations (FLOPs) measure, Multiply-add for the multiply-add 2 FLOPS
- Usage Memory , memory size (spatial complexity)
- Time Inference , inference time
- Density Accuracy , equal accuracy / modle size, used to measure the efficiency parameter
The more important conclusions:
- High computational complexity, the recognition accuracy is not necessarily high; amount parameter, the recognition accuracy is not necessarily high. - good network design is important , such as model ResNet system.
- Different parameters of different models use efficiency , higher use efficiency of network parameters for the mobile terminal now designed, such as MobileNet, ShuffleNet, SqueezeNet the like, but higher than 80% Top1 accurate model, Inception-V4 and SE- higher utilization of ResNeXt-101
- The number of operations (FLOPs) is a good estimate of the time of reasoning
- In order to meet the different memory and speed requirements, optimal model different optional
Other more detailed conclusions can be found in the paper, posted the following about the importance of chart paper.
Papers An Analysis of Deep Neural Network Models for Practical Applications born in May 2016, when the main text of the model were compared (from AlexNet to Inception-v4), has been widespread goes ball chart. Later in September 2018, article author Eugenio Culurciello in blog Analysis of deep neural networks in, for chart has been updated to include a comparative analysis Shufflenet, Mobilenet, Xception, Densenet, Squeezenet and other recent models, updating ball chart as follows:
FIG., The center position of the blob in the chart model, the parameters of the model corresponding to the amount of the size of the blob, the horizontal axis is the operating frequency and the vertical axis represents the accuracy of the Top-1 center crop, computational complexity of the model closer to the upper left corner the lower, the higher the accuracy, the smaller blob fewer model parameters .
Papers An Analysis of Deep Neural Network Models for Practical Applications , the relationship between time and number of charts reasoning follows, not surprisingly positive correlation
Paper Benchmark Analysis of Representative Deep Neural Network Architectures , the comparison made in more detail, as shown below, the model-based robust performance ResNet upper left corner, upper right corner of accuracy NASNet-A-Large maximum computational complexity but also the maximum :
Utilization parameters as follows:
Speed (frame rate) with the following accuracy, curve in the figure is the upper bound of the lower frame rate and properties of the specific hardware, the horizontal axis represents the number of frame rate,
Model parameters as the size of the amount of memory usage, minimal memory footprint on the GPU are above 0.6G,
For specific reasoning time and memory usage of each network can be found in the original paper, there are described in more detail.
Given hardware platform, the optimal model at different speeds and memory constraints are as follows:
Benchmark the Analysis of the Network Architectures, Neural Representative Deep code based pytorch, see Models-comparison.pytorch .