Return to Deep Learning Network model development history of classic catalog
Previous: depth articles - classic History of Deep Learning Network model (d) elaborate structure and features VGG16
Next: depth articles - classic History of Deep Learning Network model (six) elaborate structure and characteristics of ResNet
In this section, GoogleNet elaborate structure and characteristics (including v1, v2, v3), the next section elaborate structure and features ResNet
GoogleNet 论文：Going deeper with convolutions
II. Network Classic (Classic Network)
5. GoogleNet (including v1, v2, v3)
In the 2014 classification task ImageNet beat VGG-Net won the championship, its strength is certainly very strong. GoogleNet with AlexNet, VGG-Net rely solely on this network structure and improve network performance is not the same as thinking that open up quiet lane, while deepening the network (22 floors), and also made innovation on the network structure, the structure of the introduction of Inception instead of the simple convolution + activation function legacy operation (which the idea first proposed by the Network in Network). GoogleNet further put onto a new level of research convolution neural network.
(1). Inception V1 model
. ① V1 Network Graphics:
4 feature map convolution of high and constant width, and then up through the concatenation channels.
②. Inception V1 model variants (used in the paper is that GoogleNet)
V1 variant Network Graphics
feature map of the convolution of high and constant width, and then the concatenation up through channels.
. ③ GoogleNet features:
a. GoogleNet Inception modules 9 using a structure of 22 layers.
b. avg_pooling network instead of using the last fully connected layer, from the idea of NIN, was only AlexNet of parameters , performance is better than AlexNet. TOP1 accuracy can be proved by 0.6%, but, actually finally added a full connection layer, mainly in order to facilitate future do fine-tuning.
c. While the whole part of the connection layer is removed, but the network is still used Dropout.
D. In order to avoid the disappearance of the gradient, an additional network of two auxiliary softmax gradient for forward conduction. The two auxiliary classifier loss should be added a attenuation coefficient.
. E GoogleNet main ideas:
(A). Depth, deeper layers
(b). width, a variety of nuclear increased , and directly max pooling. To avoid excessively large thickness feature map, respectively plus the time concate
convolution kernel to reduce the thickness of the feature map.
. F Disadvantages:
(A). Too many parameters, if limited training data set, is easy to over-fitting.
(B). The larger the network, the greater the computational complexity is difficult to apply.
(C). The deeper the network, the more the gradient back pass more easily dispersed, difficult to optimize the model.
④. GoogleNet configuration diagram
(2). Inception V2 model
①. V2 Network Graphics
②. V2 features
a. V2 V1 on the basis of learning VGG-Net, two convolution kernel instead of the
large convolution kernel, and proposed methods BN, BN used for each layer.
b. the large size of the convolution kernel can bring greater receptive fields, but also means more parameters. For example convolution kernel parameter is the
convolution kernel parameters
times.
c. The two convolution instead of
a large convolution kernel, while maintaining the scope of the receptive field, while reducing parameters, and reduce over-fitting.
d. BN (Batch Normalization, BN) is a very effective method of regularization, allowing large-scale training speed network acceleration convolution many times, while the classification accuracy after convergence can be improved significantly.
e. removing partial response normalized LRN (Local Response Normalization, LRN)
f. V2 by removing the LRN, to increase BN, increasing the learning rate, the training data more thoroughly shuffle operations, etc., at 14 times faster to reach the accuracy of V1 during training, and higher accuracy of the model at the convergence limit .
(3). Inception V3 model
①. V3 Network Graphics
Another feature of the network pattern ②. V3 of
③. V3 characteristics
V3 on the basis of V 2, the introduction of small convolution factorization thinking, a large two-dimensional convolution split into a smaller two-dimensional convolution (as will be split into
and
). Thus while saving a number of parameters, calculating the acceleration and slowing overfitting, on the other hand adds a layer of non-linear, expression model extension, we can handle more and more feature rich space, increase the diversity of characteristics.
Return to Deep Learning Network model development history of classic catalog
Previous: depth articles - classic History of Deep Learning Network model (d) elaborate structure and features VGG16
Next: depth articles - classic History of Deep Learning Network model (six) elaborate structure and characteristics of ResNet
