Finishing depth learning concepts

Finishing depth learning concepts

Analytical gradient and gradient values

Gradient value is calculated by the gradient to define the gradient obtained: administering x_i a very small offset, to calculate the shift amount y, and finally removed to x_i offset x_i is localized at an offset of y Derivative.

Analytical gradient is a direct function of the target partial derivative, obtained as partial derivatives of formula calculated gradients.

Convolution layer and pooled layer CNN

Convolutional layer is to perform two functions corresponding to the convolution operation. For example, there are image a set input, a data size of 7x7x3, assuming there are two filter, size 3x3x3, in steps of 2, (size + (n-1) * step size = 7) then I can convolution layer the output obtained with a size of n = 3 for a size 3x3x2 matrix.

This matrix can be so calculated, 3x3x1 matrix of the first M0, M0 [i, j] = X [i + m, j + n] * W0 [i, j], X is the original input matrix, W0 is the first a convolution kernel, wherein m, n represents the original input matrix X is slid to the position of the window; M1 [i, j] = X [i + m, j + n] * W1 [i, j], principle and consistent with the above, will not repeat them. Of course, after the output to be activated to perform functions Activate operation.

So why do convolution operation? Suppose we are using the full link of a hidden layer, then we need to connect each neuron and each neuron in the next layer up, if the image is relatively large, we need a lot of space to store parameters parameter. But CNN convolution layer is selected these parameters are stored in the form of Filter, so that you can save a large part of the space.

Pooling layer also use the Filter, can further reduce the amount of body hidden layer matrix.

There are sometimes, we will use the multi-layered convolution, convolution reason is many times smaller than the nucleus better than a small number of large convolution effect.