This collection of some of the neural network architecture, contains some commonly used models.
LeNet5
1.Input
input image is normalized to a uniform 32 * 32.
2.C1 convolution layer
through (1 * 5 * 5) * 6 convolution kernel, stride = 1, is generated featuremap 28 * 28 * 6.
3.S2 cell layer
through the (2 * 2) sampling nucleus, stride = 2, is generated featuremap 14 * 14 * 6.
4.C3 convolution layer
through (5 * 5 * 6) 16 * the convolution kernel, stride = 1, is generated featuremap 10 * 10 * 16.
5.S4 cell layer
through the (2 * 2) sampling nucleus, stride = 2, is generated featuremap 16 * 5 * 5.
6.C5 convolution layer
through (16 * 5 * 5) 120 * the convolution kernel, stride = 1, is generated featuremap 120 * 1 * 1.
7.F6 layer fully connected
input 1 * 1 * 120, 1 * 1 * output is 84, the number of 120 * 84 General Staff.
8.Output fully it connected layer.
Input 1 * 1 * 84 * 1 * 1 output is 10, the number of General Staff for the number of classes is 84 * 10.10 classification.
AlexNet
1.Input
input image is 227 * 227 * 3.
2.Conv1
through (3 * 11 11 *) 96 * the convolution kernel, stride = 4, (227-11) / 4 + 1 = 55, generates featuremap 55 * 55 * 96.
3.Pool1
After pooling the core 3 * 3, stride = 2, (55-3) / 2 + 1 = 27, 27 generate featuremap * 27 * 96.
4.Norm1
local_size =. 5, generates featuremap 27 * 27 * 96.
5.Conv2
through convolution kernel (5 * 5 * 96) * 256, pad = 2, group = 2 , (27 + 2 * 2-5) / 1 + 1 = 27, 27 generate featuremap 27 * 256 * .
6.Pool2
After pooling the core 3 * 3, stride = 2, (27-3) / 2 + 1 = 13, generates featuremap 13 * 13 * 256.
7.Norm2
local_size =. 5, generates featuremap 13 * 13 * 256.
8.Conv3
through (3 * 3 * 256) * 384 convolution kernel, pad = 1, (13 + 1 * 2-3) / 1 + 1 = 13, generates featuremap 13 * 13 * 384.
9.Conv4
through (3 * 3 * 384) * 384 convolution kernel, pad = 1, (13 + 1 * 2-3) / 1 + 1 = 13, generates featuremap 13 * 13 * 384.
10.Conv5
After (3 * 3 * 384) * 256 convolution kernel, pad = 1, (13 + 1 * 2-3) / 1 + 1 = 13, generates featuremap 13 * 13 * 256.
11.Pool5
through (3 * 3) of the nuclear pool, stride = 2, (13-3) / 2 + 1 = 6, is generated featuremap 6 * 6 * 256.
12.Fc6
input (6 * 6 * 256) * 4096 fully connected, generates featuremap 1 * 1 * 4096.
13.Dropout6
in training time by half so that the probability of some hidden layer output neuron is 0, so that half of the output node is lost, BP is not the time to update these nodes, the following Droupout empathy.
14.Fc7
input 1 * 1 * 4096, * 1 * 1 output is 4096, the number 4096 * 4096 General Staff.
15.Dropout7
generating featuremap 1 * 1 * 4096.
16.Fc8
input 1 * 1 * 4096, output 1000, the number 4096 * 1000 General Staff.
to sum up:
LeNet deeper than 1. Network, comprising five layers and a convolution layer 3 fully connected.
2. Use relu activation function, converges quickly to solve the problem of diffusion gradient Sigmoid appear darker when in the network.
3. Add the dropout layer, to prevent over-fitting.
4. normalized using LRN layer, create a competitive mechanism of local neuronal activity, inhibit neuronal feedback small neurons large amplification reaction, enhancing the generalization ability of the model.
5. Crop rotating within do data enhancement, the enhanced generalization ability of the model. When using the predictive picture is extracted five four corners plus the middle position and lateral tilt of a total of ten images were averaged method, which is behind the brush using basic skill game.
6. The training block, then the GPU is not so strong, Alexnet the innovative training image is divided into two separately and then combined together in a fully connected layer.
7. The overall data parameters approximately 240M.
VGG
1.Input layer
input picture is 224 * 224 * 3.
2.CONV3-64
through (3 * 3 * 3) * the convolution kernel 64 generates featuremap 224 * 224 * 64.
3.CONV3-64
through (3 * 3 * 64) * the convolution kernel 64 generates featuremap 224 * 224 * 64.
4.Max pool
via (2 * 2) max pool nucleation, generating featuremap 112 * 112 * 64.
5.CONV3-128.
After (64 * 3 * 3) * the convolution kernel 128 generates featuremap 112 * 112 * 128.
6.CONV3-128
through (3 * 3 * 128) * 128 convolution generates featuremap 112 * 112 * 128.
7.Max pool
via (2 * 2) maxpool, generates featuremap 56 * 56 * 128.
8.CONV3-256
through (3 * 3 * 128) * 256 convolution kernel, generating featuremap 56 * 56 * 256.
9.CONV3-256
through (3 * 3 * 256) * 256 convolution kernel, generating featuremap 56 * 56 * 256.
10.CONV3-256
through (3 * 3 * 256) * 256 convolution kernel, generating featuremap 56 * 56 * 256.
11.Max pool
via (2 * 2) maxpool, generates featuremap 28 * 28 * 256.
12.CONV3-512
through (3 * 3 * 256) * 512 convolution kernel to generate featuremap 28 * 28 * 512.
13.CONV3-512
through (3 * 3 * 512) * 512 convolution kernel to generate featuremap 28 * 28 * 512.
14.CONV3-512
through (3 * 3 * 512) * 512 convolution kernel to generate featuremap 28 * 28 * 512.
15.Max pool
via (2 * 2) maxpool, generating featuremap 14 * 14 * 512.
16.CONV3-512
through (3 * 3 * 512) * 512 convolution kernel, generating featuremap 14 * 14 * 512.
17.CONV3-512
through (3 * 3 * 512) * 512 convolution kernel, generating featuremap 14 * 14 * 512.
18.CONV3-512
through (3 * 3 * 512) * 512 convolution kernel, generating featuremap 14 * 14 * 512.
19.Max pool
after 2 * 2 convolution generates featuremap 7 * 7 * 512.
20.FC-4096
inputs 7 * 7 * 512, 1 * 1 * output is 4096, the number of General Staff 512 * 7 * 7 * 4096.
21.FC-4096
inputs 1 * 1 * 4096, * 1 * 1 output is 4096, the number 4096 * 4096 General Staff.
22.FC-1000
1 * 1 * input 4096, output 1000, the number 4096 * 1000 General Staff.
to sum up:
1. The parameter contains approximately 550M.
2. Use all 3 * 3 convolution kernel and the largest pool of core 2 * 2.
3 simplifies the structure of a convolutional neural network.