LSTM
LSTM network
Long short term memory, what we call LSTM, is specially designed to solve long-standing problems. All RNNs have a chain form of repeating neural network modules. In a standard RNN, this repeated structural module has only a very simple structure, such as a tanh layer.
Figure 3. RNNcell
LSTM has the same structure, but the repeated modules have a different structure. Unlike a single neural network layer, there are four here, interacting in a very special way
. Figure 4. LSTMcell
LSTM core idea
The key to LSTM is the state of the cell as a whole (the green diagram represents a cell) and the horizontal line that passes through the cell.
The cell state is similar to a conveyor belt. Run directly on the entire chain, with only a few linear interactions. It will be easy for the information to flow on it and remain unchanged.
Figure 5. Internal structure diagram of LSTMcell
If there is only the horizontal line above, there is no way to add or delete information. It is achieved through a structure called gates.
The gate can selectively let information through, mainly through a sigmoid neural layer and a point-by-point multiplication operation.
Figure 6. Information node
Each element of the output of the sigmoid layer (which is a vector) is a real number between 0 and 1, which represents the weight (or proportion) that allows the corresponding information to pass. For example, 0 means "don't let any information pass", and 1 means "let all information pass".
LSTM realizes information protection and control through three such basic structures. These three gates are input gate, forget gate and output gate respectively.
Deep understanding of LSTM
Forgotten Gate
The first step in our LSTM is to decide what information we will discard from the cell state. This decision is made through a layer called the forget door. The gate will read ht − 1 h_{t−1}ht−1 and xt x_txt, and output a value between 0 and 1 for each cell state C t − 1 C_{t-1}Ct The number in −1. 1 means "completely reserved" and 0 means "completely discarded".
Where ht−1 represents the output of the previous cell, and xt represents the input of the current cell. σσ represents the sigmod function.
Input gate
The next step is to decide how much new information to add to the cell state. This requires two steps: First, a sigmoid layer called "input gate layer" determines which information needs to be updated; a tanh layer generates a vector, which is the alternative content for updating, C^t. In the next step, we combine these two parts to update the cell state.
Now is the time to update the state of the old cell, Ct−1 is updated to Ct. The previous steps have determined what will be done, and we are now to actually complete it.
We multiply the old state by ft and discard the information that we are certain we need to discard. Then add it∗C~t. This is the new candidate value, which changes according to the degree to which we decide to update each state.
Output gate
Ultimately, we need to determine what value to output. This output will be based on our cell state, but is also a filtered version. First, we run a sigmoid layer to determine which part of the cell state will be output. Next, we process the cell state through tanh (to get a value between -1 and 1) and multiply it with the output of the sigmoid gate. In the end, we will only output the part that we determined to output.
%%
clc
clear all
close all
%加载数据，重构为行向量
num=100;
x=1:num;
db=0.1;
data =abs(0.5.*sin(x)+0.5.*cos(x)+db*rand(1,num));
data1 =data;%把你的负荷数据赋值给data变量就可以了。
%data是行向量。要是还不明白，就留言吧。
%%
%序列的前 90% 用于训练，后 10% 用于测试
numTimeStepsTrain = floor(0.9*numel(data));
dataTrain = data(1:numTimeStepsTrain+1);
dataTest = data1(numTimeStepsTrain+1:end);
%数据预处理，将训练数据标准化为具有零均值和单位方差。
mu = mean(dataTrain);
sig = std(dataTrain);
dataTrainStandardized = dataTrain;
%输入LSTM的时间序列交替一个时间步
XTrain = dataTrainStandardized(1:end-1);
YTrain = dataTrainStandardized(2:end);
%%
%创建LSTM回归网络，指定LSTM层的隐含单元个数96*3
%序列预测，因此，输入一维，输出一维
numFeatures = 1;
numResponses = 1;
numHiddenUnits = 20*3;
layers = [ ...
sequenceInputLayer(numFeatures)
lstmLayer(numHiddenUnits)
fullyConnectedLayer(numResponses)
regressionLayer];
%% WOA
lb=0.001;%学习率下限
ub=0.1;%学习率上限
% Main loop
while t<Max_iter
t
end
%将预测值与测试数据进行比较。
figure(1)
subplot(2,1,1)
plot(YTest,'gs-','LineWidth',2)
hold on
plot(YPred_best,'ro-','LineWidth',2)
hold off
legend('观测值','预测值')
xlabel('时间')
ylabel('数据值')
title('Forecast with Updates')
subplot(2,1,2)
stem(YPred_best - YTest)
xlabel('时间')
ylabel('均方差值')
title('均方差图 ' )
figure(2)
plot(dataTrain(1:end-1))
hold on
idx = numTimeStepsTrain:(numTimeStepsTrain+numTimeStepsTest);
plot(idx,[data(numTimeStepsTrain) YPred_best],'.-')
hold off
xlabel('时间')
ylabel('数据值')
title('预测图')
legend('观测值', '预测值')
figure(3)
plot(1:Max_iter,Convergence_curve,'bo-');
hold on;
title('鲸鱼优化后Error-Cost曲线图');
xlabel('迭代次数')
ylabel('误差适应度值')
Note: complete code or writing add QQ2449341593 past review
>>>>>>
[lssvm prediction] based on whale optimization algorithm lssvm data prediction matlab source code [Matlab 104 period] [prediction model 2]
[SVM prediction] based on bat algorithm improvement SVM prediction [Matlab 106] [Prediction model 4]
[SVM prediction] Gray wolf algorithm optimization svm support vector machine prediction matlab source code [Matlab 107] [Prediction model 5]
BP neural network prediction [Matlab 108] [Prediction Model 6]
[lssvm prediction model] Based on bat algorithm improved least square support vector machine lssvm prediction [Matlab 109] [Prediction model 7]