Return to Deep Learning Network model development history of classic catalog
Previous: depth articles - classic history of Deep Learning Network model development (a) graph structure and LeNet-5 structure and features about the history of the development model
Next: depth articles - classic History of Deep Learning Network model (c) elaborate structure and characteristics of the ZF-Net
In this section, AlexNet elaborate structure and characteristics, the next section elaborate ZF-Net structure and characteristics of
Papers Address: ImageNet Classification with Deep Convolutional Neural Networks
II. Network Classic (Classic Network)
2. AlexNet
In 2012 ImageNet competition, AlexNet absolute advantage over second place 10.9 percent won one fell swoop. It is often said: The year 2012 AlexNet turned out, from the image depth learning, flourishing.
(1) Network Description:
From the above network, 224x224 ~ 227x227 input is possible, since the first layer is a padding SAME
. (2) AlexNet of new features:
①. ReLU successfully used as CNN's activation function, and verify its effect on deeper network over the Sigmoid, the successful resolution of the problem is deeper in the network Sigmoid gradient diffuse. Further, to speed up the training speed, as network trained using a gradient descent method, the non-linear function of unsaturated training faster than the non-linear function of saturation and, RELU Sigmoid function is simpler than the function, the operation amount. Although ReLU activation function was proposed a long time ago, but until the AlexNet of it to flourish.
②. Dropout training using random ignored part of neurons, to avoid over-fitting model. Although Dropout separate papers said, but AlexNet to practical use, by practice confirmed its effect. In the last few AlexNet it is mainly full link layer uses Dropout.
③. CNN used in the maximum overlapping pooling. Prior to the widespread use of CNN in average pooled, AlexNet the largest pool of all use, to avoid the average pool of blurring effects. And proposes a compromise AlexNet pieces smaller than the size of the cell nucleus, so that there is overlap between the output of the pool and the cover layer, to enhance the richness of features.
④. Proposed partial response normalized LRN (Local Response Normalization, LRN) layer, to create a competitive mechanism of local neuronal activity, wherein in response to such a relatively large value becomes relatively larger, smaller and inhibit neural feedback yuan enhance the generalization ability of the model.
⑤. CUDA accelerated training depth using convolutional network, using the powerful GPU parallel computing power, when processing a large number of neural network training matrix operation. The authors AlexNet distributed on two GPU, parameters are stored in memory for each GPU half the neurons
⑥. Data enhancement
Data enhancement greatly increases the amount of data. If there is no data enhancement, relying on the original amount of data, the number of parameters CNN will fall through the fitting, the use of data enhancement can greatly reduce the over-fitting, enhance the generalization ability. When the prediction, is applied intermediate the four corners of the image taken of a total of five positions, and left-right inverted, received a total of 10 images, and they are predicted structure averaging 10 times.
Return to Deep Learning Network model development history of classic catalog
Previous: depth articles - classic history of Deep Learning Network model development (a) graph structure and LeNet-5 structure and features about the history of the development model
Next: depth articles - classic History of Deep Learning Network model (c) elaborate structure and characteristics of the ZF-Net
