100 days to get the machine learning | Day35 depth learning neural network structure

This article "What is a Neural Network?" 3Blue1Brown the study notes

Watch Address: bilibili.com/video/av15532370

This episode Grant chiefs assumptions we do not have the neural network, for novices to explain the basic concepts of neural networks, so that we again hear when the neural network learning, can understand what it really meant. Gangster choose the classic Multilayer Perceptron (MLP) structure to solve the problem of handwritten digit recognition, on the grounds that the original understanding of the classics, in order to better understand the powerful variants, such as CNN and LSTM.

First look at neurons and how they are connected

Neurons can be understood as a number between 0 and 1 filled container. * 28 to each pixel of the input image 28 as an example, each neuron is an origin, wherein the numbers represent the grayscale value of a pixel, 0 represents black, white represents 1, the figure referred to in the neural network. " activation value. "

This neuron 784 is on the first layer of the network, on behalf of the last ten neuron layer of 0-9 ten numbers, the same, their activation values are between 0 and 1 representing the input image the possibility of which number corresponds. The intermediate layer is called the network hidden layer, can be seen as a black box, the specific operation of the digital identification is completed here, there is added a two hidden layers, each with 16 neurons - easily set, but in order to appear good-looking, you can then adjust the latter.

The core mechanism of neural information processing network is the case, the value of the activation layer through certain operations, obtain the activation value of the next layer. 784 mentioned above also represents the number of neurons pattern, then the value of the next one activation also produce special patterns, the final output layer neurons represents the brightest selected neural network.

We look at how to train, why this layered structure will be able to identify the numbers?

We can digital dismantling desirable reciprocal each neuron of the second layer may correspond ideally on a stroke.

Look at the hidden layer of the first layer, when the input is above the encircled number 8, or 9, the activation value of a neuron would be close to 1, it is desirable that all images at the top of the circle pattern can be lit neurons, so that, from the third to the output layer, we just need to learn what parts which can be a combination of numbers can be.

But how to identify the circle it? Similarly you can split it into a more subtle problem.

So we hope each neural network element of the second layer corresponding to the short side, the second layer can be associated with all the shorter sides of eight to ten neurons are lit, the lighting can be followed to the top of the circle and the long bars Neurons.

Then we look at the role of the connection

How to make this a neuron in the second layer (the first layer of hidden layers) can be able to identify this one area of an image if there is an edge?

We design allows certain neurons in the second layer can correctly identify this area of the image where the existence of an edge.

We come up with the first layer (input layer / layer 0) of the activation value, and to give the weights (pay attention to the area of heavy weights is positive, 0 otherwise), so seek weighted and it will only be cumulative for all the pixel values the value of the pixel area of concern.

Green is positive and red is negative, the darker the color represents the weight closer to 0

So if you want to identify whether there is an edge, just give a lap around the pixel is given a negative weight

Weight values can be made of any size

But what we need is to be compressed to between 0-1

Here it is necessary Sigmoid function, and this function is activated.

Activation value is actually a weighted sum of scoring positive in the end how much, but sometimes the weighted sum is greater than 0, do not want to light up neurons, such as weight and if you want to let him inspire greater than 10, there will also need to plus an offset value, to ensure that not just excited.

Tell us the weight of the second neuron concerned about what kind of pixel pattern

bias weighted and tell us how much you want to stimulate neurons to make meaningful

There are more than 13,000 parameters need to be adjusted, so-called learning here is to find the right weight and bias. We talked about the beginning of neurons seen as a digital container, but these numbers depend on the input image, so the neurons seen a function is more accurate, it is a layer of input neurons of all output, its the output is a value between 0-1. In fact, the whole neural network is also a function of the input 784, the output value 10. But it is a weight contains 13000, is extremely complex offset parameter of the function.

It should be noted, in particular, machine learning and neural network computing linear algebra are inextricably linked, have been introduced before, the venue:

100 days to get the machine learning | nature Day26-29 linear algebra

The neural network requires a large amount of matrix multiplication and mapping operations sigmoid

This calculation method, programming is very suitable, such as using the Python numpy very simple implementation.

Finally, it should be mentioned is that now no longer used sigmoid basic neural network, the more popular is ReLU (rectified linear function).

100 days to get the machine learning | Day35 depth learning neural network structure

Guess you like