Today, two major share Github project, are based on PyTorch to achieve deep learning network model, mainly because some popular models, including such ResNet, DenseNet, ResNext, SENet, etc., and also gives the corresponding experimental results, contain complete data handling and loading, model building, training process to build, test and implement the code.
Then we began to introduce these two projects.
(I want to learn programming from a small partner search circle T community , more and more industry-related industry information about free video tutorials. Oh, absolutely free!)
1. PyTorch Image Classification
The Code currently has 200+ star, the main achievement of the following network, carried out experiments on MNIST, CIFAR10, FashionMNIST and other data sets.
Use as follows: Then is the author's own experimental results of training, and then the experimental results of the comparison of the original paper, including differences in the training set, then the number of iterations of training and training time are given respectively.
After the authors also studied the residual unit, learning strategies and data rate enhancement effect on the classification performance, such as
- Residual pyramid-like network cell design (PyramidNet-like residual units)
- Cosine function of decreasing the learning rate policy (Cosine annealing of learning rate)
- Cutout
- Eliminate random (Random Erasing)
- Mixup
- Pre-activation shortcut (Preactivation of shortcuts after downsampling) after downsampling
Experimental results show:
- Similar pyramid network of residual cell design is helpful, but not suitable for matching Preactivation of shortcuts after downsampling
- Based on the learning rate decreasing cosine strategies to enhance the rate is small
- More Cutout, Mixup and eliminate random effects are good, which Mixup required training times
In addition to this experiment, the authors also behind continue to do several experiments, including batch size, the size of the initial learning rate, smoothing label has done a lot of experiments and experimental results corresponding.
Finally, the paper models of these networks, the paper of different training strategies.
In addition to implementing this project the corresponding network model, the use of different techniques or basic research of batch size, initial learning rate is can give us some inspiration, some tricks that can be applied to the network, improve the classification performance.
Links:
github.com/hysts/pytor...
2. PyTorch Image Models
The Code currently has 600+ star, and in recent days have also updated the network to achieve more, including DPN, Xception, InceptionResNetV2, and the recent comparison of fire EfficientNet.
This project has the following characteristics:
- All models have done a package, that is, it has a default configuration interface and API, including unified call classifier interfaces
get_classifier、reset_classifier, forward calculation of the characteristics of the interfaceforward_features - We have a consistent model pre-trained model loader, which can decide whether to use or whether the last one of the input layer pre-trained channel model requires from 3 to 1 channel;
- Training script can be used in different modes, including a distributed, multi-card single, stand-alone single card or stand-alone CPU
- To achieve dynamic operation of the cell layer, comprising pooled mean (average pooling), the maximum pooled (max pooling), the mean + the maximum, average and maximum or pooled result of the connection rather than superimposed;
- Implement different training strategies, such as cosine learning rate, the elimination of random, smoothing labels
- Achieve Mixup
- Provide a forecast script
The same author gives the results of training, and then there use, is also ready for data, you can use at the specified location.
In addition, the author also gives ToDo list, we will continue to improve the project.