- Neural network training process can be divided into three steps
Output structure and the forward propagation neural network 1. Definitions
2. Define the function and selecting a reverse propagation loss optimization algorithm
3. Build and run repeatedly in the session on the training data back-propagation algorithm optimization
Neural Networks
Neurons
Neurons are the minimum unit constituting the neural network neurons is structured as follows:
A neuron has multiple inputs and one output, the input of each neuron can output either other neurons may also enter the entire neural network.
All simple output neuron is shown in FIG weighted sum, i.e., input different weights of all neurons of the input neurons is the weight parameters (Wi), the optimization process is to optimize the neuron Parameter of neurons
process.
Fully connected neural network
Fully connected neural network between any two nodes are connected to the adjacent layers.
Three fully connected neural networks:
Wherein the input from the physical layer was extracted feature vectors, hidden layers, the more complex neural network structures. DNN means one based on the collection of high complexity data modeling algorithm through a multilayer non-linear transformation.
Forward propagation
Forward propagation is the output layer through the hidden layer is a calculation process to obtain the final output layer.
Forward propagation:
x1 and x2 for the input of the neural network, the neurons W represents the parameters, the layers labeled neural network, the connection node numeral subscripts, the value W on the edge of the current weight.
Since the output of the neural network input neurons and weighted, so
a11:
Y:
Activation function
The above-described relationship model is a simple linear model, the input and output satisfy the linear model
wi, b∈R are parameters of the model, only one time when the input x and y to form a straight line of a two-dimensional coordinate system. When the input n is a plane n + 1-dimensional space.
But in the real world, most of the problems are not linear segmentation. If we use a linear model, the output signal will only be a simple linear function.
We need the help of a nonlinear function of helping us to understand and learn from other complex types of data, this function is to activate the function.
Linear activation function to
If the output of each neuron through a non-linear function, the entire neural network model is no longer a linear.
Where y is to the process A by linear activation function.
Loss function
In supervised learning, it is necessary to measure the magnitude of the difference between the neural network output and expected output. This error function needs to be able to reflect the current degree of inconsistency between the network after a quantized output and the actual results, that is to say the larger the function value reflects the results of the prediction model inaccurate. This loss function is the error function.
The effect of neural network model and optimization goal is defined by loss of function.
Thoughts on supervised learning is known answer labeling data sets, model results are given as close to the real answer. By adjusting the parameters of the neural network
to fit the training data, so that the model provides the ability to predict unknown samples.
Back-propagation algorithm
Back-propagation algorithm implements an iterative process, each iteration beginning, take the first part of the training data, through forward-propagation algorithm
Get prediction neural network. Because the training data have the correct answer, it is possible to calculate the difference between the predicted results and the correct answer.
Based on this difference, the value of the corresponding back-propagation neural network algorithm parameters will be updated so that the real answers and closer.
Reverse flowchart propagation algorithm:
Popular understanding is that we played Bouncing Church.
We fired artillery shells by adjusting the angle and intensity to hit the target, assuming no wind affect other external factors.
After he missed the first time we launch a fight in front of the goal, the second time we will properly intensifying the downward adjustment of the angle, after repeated several times that we will grasp the right intensity and angle.
Every hit point and the goal difference is the error, you can use a function to represent the loss.
After each launch point of view hitting position, and then adjust the angle and intensity of this process is the back-propagation.
Gradient descent algorithm
Step mainly descent algorithm for optimizing a single parameter value, the back propagation algorithm is an efficient way to give a gradient descent algorithm on all parameters.