How to determine the weight of neural network?
The weight of the neural network is obtained by training the network. If you use MATLAB, don’t set it yourself, it will be automatically assigned after newff. It can also be done manually: {}= ; {}=. Generally speaking, the input is normalized, then w and b take random numbers from 0-1.
The purpose of determining the weight of the neural network is to allow the neural network to learn useful information during the training process, which means that the parameter gradient should not be 0.
Parameter initialization must meet two necessary conditions: 1. There will be no saturation in each activation layer. For example, for the sigmoid activation function, the initialization value cannot be too large or too small, causing it to fall into its saturation zone.
2. Each activation value is not 0. If the output of the activation layer is zero, that is, the input of the next convolutional layer is zero, so the partial derivative of the weight of this convolutional layer is zero, resulting in a gradient of 0. Extended information: the relationship between neural networks and weights.
When training an agent to perform a task, a typical neural network framework is chosen with the belief that it has the potential to encode a specific policy for that task. Note that there is only potential here, and weight parameters must be learned to change this potential into ability.
Inspired by pre-existing behaviors and innate abilities in nature, in this work, the researchers constructed a neural network that can naturally perform a given task. In other words, find an innate neural network architecture, and then only need randomly initialized weights to perform tasks.
The researchers said that this neural network architecture that does not need to learn parameters performs well in both reinforcement learning and supervised learning. In fact, if you imagine that the neural network architecture provides a circle, then the conventional learning weight is to find an optimal point (or optimal parameter solution).
But for a neural network that does not need to learn weights, it is equivalent to introducing a very strong inductive bias, so that the entire architecture is biased to directly solve a certain problem. But for a neural network that does not need to learn weights, it is equivalent to continuously specializing the architecture, or reducing the variance of the model.
In this way, when the architecture becomes smaller and only contains the optimal solution, the randomized weights can also solve practical problems. It is also feasible to search from a small architecture to a large architecture like the researchers, as long as the architecture can just surround the optimal solution. Reference source: Baidu Encyclopedia - Neural Network.
Google AI Writing Project: Neural Network Pseudo-Original
How to determine the connection weight of BP neural network
Confirmation method: Statistics believe that when calculating the average and other indicators in statistics, the value that has a weighing effect on the value of each variable is called the weight .
Example: To find the average of the following number strings 3, 4, 3, 3, 3, 2, 4, 4, 3, 3, the general method is (3+4+3+3+3+2+4+4+3+3)/10=3.2 The weighted method is (6*3+3*4+2 )/10=3.2 Among them, 3 appears 6 times, 4 appears 3 times, and 2 appears 1 time. 6, 3, 1 are called weights.
This method is called weighting method. Generally speaking, the average is to add up all the numbers and divide by the total number of these numbers.
Expressed as: (p1+p2+p3+…..+pn)/n; But some data records have some identical data, when calculating, if there are several identical numbers in that number, multiply this number by A few, this few are called weights, and weighting means multiplying by a few and then adding.
The average is still divided by the total.
Still take the above numbers as an example: each of them has some same numbers, expressed as: k1, k2, k3...kn; the formula for weighted average is: (k1p1+k2p2+k3p3+...knpn)/( k1+k2+k3+.....kn).
Setting of Initial Weight and Threshold in BP Neural Network
1. First of all, you need to understand that the BP neural network is a multi-layer feedforward network. 2. Take a look at the training function of BP neural network in matlab, there are gradient descent method traind, elastic gradient descent method trainrp, adaptive lr gradient descent method trainda, etc.
3. Define the input P and output T in the command line window in matlab, and construct the BP neural network through "newff(minmax(P),[5,1], "[net,tr]=train(net,P,T );" to train the network, and "sim(net,P)" to get the simulation prediction value.
4. After pressing the Enter key in the command line window, you can see the result pop-up window. The Neural Network on the top represents "input, hidden layer, output layer, output" in sequence, and there are 5 hidden layers Neurons.
5. The Epoch under Progress represents the number of iterations, Gradient represents the gradient, Vaildation Checks represents the validity check, and the final green tick represents the achievement of the performance goal.
6. Finally, draw the actual curve and the predicted curve, and you can see that the result curve predicted by the BP neural network is basically consistent with the actual output curve.
Setting of Initial Weight and Threshold in BP Neural Network
1. First of all, you need to understand that the BP neural network is a multi-layer feedforward network. 2. Take a look at the training function of BP neural network in matlab, there are gradient descent method traind, elastic gradient descent method trainrp, adaptive lr gradient descent method trainda, etc.
3. Define the input P and output T in the command line window in matlab, and construct the BP neural network through "newff(minmax(P),[5,1], "[net,tr]=train(net,P,T );" to train the network, and "sim(net,P)" to get the simulation prediction value.
4. After pressing the Enter key in the command line window, you can see the result pop-up window. The Neural Network on the top represents "input, hidden layer, output layer, output" in sequence, and there are 5 hidden layers Neurons.
5. The Epoch under Progress represents the number of iterations, Gradient represents the gradient, Vaildation Checks represents the validity check, and the final green tick represents the achievement of the performance goal.
6. Finally, draw the actual curve and the predicted curve, and you can see that the result curve predicted by the BP neural network is basically consistent with the actual output curve.
How to adjust the initial value of the threshold and weight in the neural network? Why do I always have a particularly large error?
What do neural network weights mean?
The weight of the neural network is obtained by training the network. If you use MATLAB, don’t set it yourself, it will be automatically assigned after newff. It can also be done manually: {}= ; {}=. Generally speaking, the input is normalized, then w and b take random numbers from 0-1.
The purpose of determining the weight of the neural network is to allow the neural network to learn useful information during the training process, which means that the parameter gradient should not be 0. A network is composed of several nodes and links connecting these nodes, representing many objects and their interrelationships.
Before 1999, people generally believed that the structure of the network was random.
But after Barabasi and Watts discovered the scale-free and small-world properties of the network in 1999 and published their findings in the world-renowned "Science" and "Nature" magazines, people realized the complexity of the network. sex.
The Internet will bring people an extremely rich life and wonderful enjoyment in terms of text, pictures, sound, and video with the help of software tools such as text reading, picture viewing, video and audio playback, downloading and transmission, games, and chatting.
In Chinese, the word "network" was first used in the "Modern Chinese Dictionary" (1993 edition) of electricity to make such an explanation: "In an electrical system, a circuit composed of several components is used to transmit electrical signals according to certain requirements. Or part of this circuit, called the network.
"In mathematics, a network is a kind of graph, which is generally considered to refer to a weighted graph. In addition to the mathematical definition, the network also has a specific physical meaning, that is, the network is a model abstracted from a certain type of practical problem.
In the computer field, the network is a virtual platform for information transmission, reception, and sharing. Through it, the information of various points, surfaces, and bodies is linked together, so as to realize the sharing of these resources. The network is the most important invention in the history of human development, which has improved the development of technology and human society.
How to set the initial weight of BP neural network
About the weight training method of neural network
% Read in training data and test data Input = [];Output = [];str = {'Test','Check'};Data = textread([str{1},'.txt']);% Read training Data Input = Data(:,1:end-1);% Get the first five columns of the data table (master-slave components) Output = Data(:,end);% Get the last column of the data table (output value) Data = textread ([str{2},'.txt']);% Read test data CheckIn = Data(:,1:end-1);% Get the first five columns of the data table (master-slave components) CheckOut = Data(:, end);% Get the last column (output value) of the data table Input = Input';Output = Output';CheckIn = CheckIn';CheckOut = CheckOut';% Matrix [Input,minp,maxp,Output,mint,maxt ] = premnmx(Input,Output);.
How to correct the weight of the wavelet neural network and the coefficient of the wavelet basis function according to the prediction error?