Original article, reprint please indicate that it is from "Old Cake Explains Neural Network": bp.bbbdata.com
Table of contents
3. Initialization instructions
(2) Initialization of W32 weights
4. Description of training rules (LVQ1, LVQ2)
This article explains the specific algorithm flow and details of LVQ. The algorithm flow is taken from the matlab2014 neural network toolbox, which is the specific explanation of "LVQ-reproduce matlab implementation code".
1. Description of symbols
- W21: Represents the weight matrix from the first layer (input layer) to the second layer (hidden layer) of the network. size: number of hidden neurons * number of inputs. Each i row represents the weights of all inputs to the ith neuron.
- W32: Represents the weight matrix from layer 2 (input layer) to layer 3 (hidden layer) of the network. size: number of outputs * number of hidden neurons. Each i row represents the weights of all hidden neurons to the ith output.
2. Main process
The training of LVQ mainly uses the single-sample training method, and its main process is as follows:
- 1. Initialize the weights w21 and w32.
- 2. Loop iterative training until the exit condition is reached (the error is small enough or the maximum number of training times is reached)
- ---- Randomly select N samples without replacement, and use one sample for training each time:
- --------- Calculate the hidden node update amount dw21 according to the training rule lvq1 or lvq2.
- ---------Update w21 = w21+dw21
- ----Calculate the error of this round, if it is the best in history, take the current w21 as the best w21 in history.
- Return the best w21 in history.
The training flow chart is as follows:
3. Initialization instructions
(1) W21 weight initialization
W21 is the weight from the input layer to the hidden layer, which is initialized by the center point.
That is, all hidden neurons are initialized to the center point of the input, namely
W21 weight initialization calculation example:
2 inputs, 3 hidden nodes.
Input range for X:
Then
(1) Calculate the center points of x1 and x2 as 5 and 4 respectively.
(2) Then W21 is initialized as
(2) Initialization of W32 weights
W32 is not directly set, it is initialized by the proportion of the connection between the output node and the hidden node pc.
For example, 3 hidden nodes, 2 outputs, when pc = [0.4,0.6], the first node occupies floor(0.4*3) = floor(1.2)=1 hidden node, and the second output occupies 2 indivual.
PASS: In terms of code implementation, first calculate the cumulative value of pc, and then map the cumulative value to the end position of the hidden neuron number, so as to determine the specific allocation of hidden neurons, and then determine W32.
W32 calculation example explanation:
the number of hidden nodes hn: 8, the output number out_n = 3, the distribution ratio pc=[0.3,0.2,0.5]
(1) Calculate the cumulative value of pc
(2) Mapping to the end position of hidden neuron number
It can be obtained that hidden nodes 1-2 are connected to output 1, hidden nodes 3-4 are connected to output 2, and hidden nodes 5-8 are connected to output 3.
(3) Determine W32
4. Description of training rules (LVQ1, LVQ2)
(1) LVQ1
(1) Initialize dw21 to be all 0 (the size is consistent with w21)
(2) If the sample used for training (note that there is only one sample at a time) is predicted correctly, the hidden neuron weight (position) adjustment for successful competition is sample direction, otherwise it is the opposite direction of the sample.
illustrate:
- k : The hidden neuron number of the successful competition.
- dW21(k,:) : represents the kth row of dW21.
- lr: learning rate, which can be set to 0.01,
- x: the input sample for this training,
- w: The weight corresponding to the successful hidden neuron, that is, the kth row of W21.
(2) LVQ2
(1) Get the numbers k1 and k2 of the nearest cluster center point and the next closest cluster center point
(2) If the distances d1 and d2 from the training sample to k1 and k2 are too small, that is, satisfy
is updated.
5. Network Prediction
The LVQ network output y can be directly calculated according to the mathematical expression:
Among them, dist is the Euclidean distance, and compete is the vector competition function, that is, the maximum value in the vector is 1, but it is actually 0.
related articles
"BP Neural Network Gradient Derivation"
"Mathematical Expressions Extracted by BP Neural Network"
"A Complete Modeling Process of BP"