Advanced convolution neural network
The limitations of using fully connected layers:
- The image in the same column adjacent pixels may be far apart in this vector. They constitute a pattern recognition model may be difficult.
- For the large size of the input image using a full model of the connection layer easily lead to too large.
The advantage of using convolution layer:
- Convolution input shape retention layer.
- Convolution of the input layer is formed by a sliding window with the same convolution calculation is repeated at different locations to avoid oversized parameters.
LeNet
LeNet layer blocks into two parts and the convolution block layer fully connected, then the alternating layers and the maximum pool convolution layers fully connected layers for image classification. Below we describe these two modules.
Layer block convolution in the basic unit cell layer average access layer convolution: Convolution layer serves to identify the image in the spatial patterns, such as lines and local object, the average cell layer after layer is used to reduce the convolution the sensitivity of the location.
Layer block convolution of two such basic units are repeatedly stacked configuration. Layer block convolution, convolution in each layer uses \ (5 \ times 5 \) of the window, using sigmoid activation functions and output. The first convolutional layer output channels is 6, a second convolutional layer is increased to 16 the number of output channels.
Fully connected layer block contains 3 layers fully connected. The number of outputs thereof respectively 120,84 and 10, where 10 is the number of output classes.
LeNet: performance on large data sets do not do real as Face meaning.
1. The neural network computational complexity.
2. START yet deep research zoomed amount parameter initialization and comes in handy convex optimization algorithms and many other areas.
AlexNet
For the first time proved that learning can go beyond the features of the watch ⼯ design features so ⼀ before breaking move like computer vision research.
feature:
- Conversion layer 8, which has a convolution layer 5 and the layer 2 hidden layer fully connected, and an output layer fully connected.
- The sigmoid activation function into a simpler ReLU activation function.
- Dropout used to control the whole connection layer model complexity.
- The introduction of data enhancement, such as flipping, cropping and color change, thus further expanding the data set to ease over-fitting.
Duplicate network element (VGG)
VGG: by repeatedly making a simple base block constructed Use depth model.
Block: the same number of filled 1, shape of the window \ (3 \ times 3 \) convolution layer, connected to a stride of 2, shape of the window \ (2 \ 2 Times \) Maximum cell layer .
Convolution layer remains unchanged input height and width, and the cell layer is its half.
Open networks in Open networks (NiN-)
LeNet, AlexNet and VGG: a module composed of a first convolutional layer is sufficiently extracted feature space, then the module is connected through a full layer classification result is output.
NiN: convolution layers and a plurality of series "fully connected" small Open networks constructed ⼀ layer made of a deep Open networks.
Using the number of output channels equal to the number of blocks NiN category label, then use the average global pool of all elements of the layer averaged in each channel to directly Use classification.
1 × 1 nucleation convolution
1. Grading channels: By controlling the number of channels reaches a convolution kernel of zoom.
2. Increase nonlinear. 1 × 1 convolution process equivalent to convolution calculation process fully connected layers, and also adding a nonlinear activation function, thereby increasing the non-linear network.
3. Calculation fewer parameters
Repeatedly using the convolution NiN layer and a connecting layer instead of whole blocks of 1 × 1 NiN layer composed of convolution to build deep Open networks.
NiN removed easily cause the output layer through the full connection fitting, but rather to replace it with the number of output channels equal to the number of blocks and a global category label NiN average cell layer.
Open networks to a wide range of design convolution nerve after NiN the above design ideas have influenced the ⾯.
GoogLeNet
- Inception by the base blocks.
- Inception ⼀ a block corresponding to a line with a 4 Submenu Open networks. And ⾏ it to extract information layer and a convolution layer is largest pools of different window shapes, and use a 1 × 1 convolutional layer reduces the number of channels to reduce the complexity of the model.
- Custom ultra parameter is the number of output channel of each layer, in order to control the complexity of our model.
Complete model structure
