Convolution neural network
Convolution neural network consists of the following five structures
- The input layer. Is input across the input layer of the neural network, the neural network convolution process in the image, which generally represents a matrix of pixels of a picture. Starting from the input layer, a convolutional neural network through a different neural network structure on the layer of three-dimensional matrix into the next layer of three-dimensional matrix, until finally the entire connection layer.
- Convolution layer. Can be seen from the name, is a convolution convolution layer neural network is the most important part. And the conventional fully connected layers of different convolutional layers each node in the input layer of only a small neural network, this small size have common 3 × 3 or 5 × 5. Each tile has tried to convolutional neural network in more in-depth analysis to obtain a higher level of abstraction features. The treated matrix by convolution node will become deeper layers.
- Cell layer (Pooling). Pooling layer neural network does not change the depth of the three-dimensional matrix, but it can reduce the size of the matrix. Pool operation can be considered to be a higher resolution image into a lower resolution picture. Finally, narrowing the number of nodes in a fully connected layer.
- Fully connected layer. After several rounds of treatment and the pooled layers convolution layers, usually the last convolution neural network is a fully connected one to two layers to give the final classification result. After several rounds of treatment and the pooled layers convolution layer, may be considered in the information image has been abstracted become higher information content of the feature. We pooled convolution layer and layer as the image feature extraction process automatically. After feature extraction is complete, still need to use fully connected layer to complete the classification task.
- Softmax layer. And as described earlier, Softmax layer primarily used for classification. By Softmax layer can be obtained where the probability distribution of a current sample belongs to a different species.
Convolution layer
The filter may be a child node on the current matrix layer neural network is converted to a unit matrix of nodes on the next layer neural network. Refers to a unit matrix is a node length and width are both 1, but not the depth of the node matrix.
The pre-filter by the propagation process is left submatrix is the right node calculation unit matrix of nodes.
The following equation demonstrates how the filter is a matrix node changes a 2 × 2 × 3 is a
unit matrix of nodes of 1 × 1 × 5. i represents the i-th node of the output node proof unit, here are the 5, so there g (0) ~ g (4 ) of the output filter through the input node (x, y, z) of the weight, using B I to paranoid parameters is such that g (i) have the following formula
again a g (0) is out how, as the depth of a given node is three, then the depth of the filter is also given, and the depth is the same III. a and w. Such dot can be calculated g (0) is the value of. Since the depth of the node layer of the unit is 5, so there is g (1) ~ g (4 ) output, but also have a different weight is calculated.
Calculating a, there are so many of it, the mobile computing thousand million in the filter plane. Therefore, before the layer structure of the convolution process it is to propagate through a filter the upper left corner of the mobile from the neural network to the lower right of the current layer, and calculates the movement corresponding to each of the resulting matrix. These obtained values spliced into a new matrix, to complete the process of propagating convolutional layer.
This is the filter parameter sharing. (With a filter to move the entire plane transformation parameters it is it still?)
Convolutional neural network, each of the convolution filter layer used in the parameters are the same. This is the convolution neural network a very important property. The number of parameters independent of the size of the image and the convolution layer, only the size and depth of the filter, and the depth of the current layer node matrix. This makes the convolutional neural network can be expanded well to a larger image data.
This weighting filter parameters not say it
Pooling layer
Cell layer is very effective in reducing the size of the matrix (typically mainly reduce the length and width of the matrix, the depth is not required), thereby reducing the final parameters fully connected layer. Use pooling layer can be computed faster but also to prevent over-fitting problem of the role.
And convolution-like layer, the front layer of the cell propagation process is similar to moving a filter accomplished by the structure. However, pooled weighted filter layer nodes and not, instead of using a maximum value or an average of more simple operation.
Using the maximum value of the operation layer is referred to the maximum pool pooled layer (max pooling), which is the most used cell layer structure. Pool using the average value of the operation layer is referred to the average cell layer (average pooling).
Using convolution filter layer across the entire depth of the pool and the filter layer used only affect a node on the depth.
Transfer learning
As technology advances, the convolution model more complex, they require training or special labeling data and more, and even if there are vast amounts of training data to train a neural network convolution complex also take days or even weeks. Then there is the migration study, is to study the migration problem on a trained model by simple adjustments make it suitable for a new problem.
Called bottleneck layer (flying - by -) before finally the entire network layer connection layer this layer. The new image by convolutional neural network trained until the process can be seen as a bottleneck layer processes the image feature extraction.
other
The classic picture convolution neural network architecture for classification problems: the
input layer → (? + → convolution layer pooled layer) layer fully connected + → +
