1. A simple neuron
The picture shows a simple neuron.
2 inputs: X1, X2;
1 output: Y.
w1, w2 are weight weights, and b is bias deviation;
The importance of the bias is: the main function is to provide a trainable constant value for each node (except for the regular input that the node accepts)
After this neuron, it is processed as w1 X1+w2 X2+b;
after another activation function f , the output is formed.
The activation function f is usually a non-linear function, and the purpose is to introduce non-linearity into the output of the neuron. Because most real-world data is non-linear, it is hoped that neurons will learn such a non-linear form.
The activation function mainly introduces 3:
1. sigmoid: compress the input value into a number between 0 and 1.
The diagram is as follows:
2. tanh: compressed to [-1,1].
The diagram is as follows:
3. Relu: If the input is a negative value, then =0; if the input is a non-negative value, it remains unchanged.
The figure is as follows:
4. softmax: For classification problems, softmax function is generally selected as the activation function to ensure that the output represents the probability, and the sum is equal to 1.
2. Feedforward neural network
Simple example of a feed-forward neural networks as shown:
having an input layer: 3 neurons, corresponding to the three input
a hidden layer: 3 neurons
an output layer: 2 neurons, two outputs corresponding to
two Examples of feedforward networks:
1. Single-layer perceptron: no hidden layer
2. Multi-layer-pereptron (MLP): one or more hidden layers
One hidden layer can learn linear functions, and multiple hidden layers can learn linear and nonlinear functions.
Given a set of features X=(x1,x2,...) and the target y, the multilayer perceptron can learn the relationship between the features and the target for classification or regression.
3. How to train MLP: Backpropagation algorithm
For example, supervised learning: the training data are all labeled, that is, both input and output are given. After the input passes through the neural network, there will be an output, which is compared with the correct output and learns from mistakes. If an error occurs, it can be corrected by modifying the value of the parameter.
Initially, all edge weights are randomly assigned values. For each input in the training data set, the neural network is activated and the output is observed, and the output is compared. The error is backpropagated to the upper layer, the error is recorded and the weight is adjusted, and the process is repeated until the output error is lower than the predetermined threshold. (That is, the error of the error is the acceptable range I set)
Use backpropagation to calculate the gradient and pass the error back to the network. Use optimization methods such as gradient descent to adjust the weights and reduce errors. Repeat the above steps for each training data, and finally the network will learn.
For the test set, input the data, propagate forward, and get the result.
Learning source: https://ujjwalkarn.me/2016/08/09/quick-intro-neural-networks/