[Time series forecasting] Single-feature electricity load forecasting based on BP, LSTM, CNN-LSTM neural network algorithm [nanny-level hands-on teaching]

Series Article Directory

Principles of deep learning ----- linear regression + gradient descent method Principles
of deep learning ----- logistic regression algorithm
Principles of deep learning ----- fully connected neural network Principles
of deep learning ----- convolutional neural network
depth Learning principle-----recurrent neural network (RNN, LSTM)
time series forecasting-----based on BP, LSTM, CNN-LSTM neural network algorithm, single-feature electricity load forecasting
time series forecasting (multi-features)-- ---Multi-feature electricity load forecasting based on BP, LSTM, CNN-LSTM neural network algorithm

---

Series of teaching videos

Quick introduction to deep learning and actual combat
[hands-on teaching] based on BP neural network single-feature electricity load forecasting
[hands-on teaching] based on RNN, LSTM neural network single-feature electricity load forecasting
[hands-on teaching] based on CNN-LSTM neural network single-feature electricity consumption Load forecasting
[Multi-feature forecasting] Multi-feature electric load forecasting based on BP neural network
[Multi-feature forecasting] Multi-feature power load forecasting based on RNN and LSTM [Multi-feature forecasting
] Multi-feature power load forecasting based on CNN-LSTM network

---

foreword

  Electric energy is an energy that human beings cannot do without, but electric energy is an energy that is not easy to store. Most of China's current electric energy is generated by thermal power; the consumption of electric energy in a region changes with time. Change; if the power generation is lower than people's power consumption, it will lead to voltage instability and many people will not be able to use electricity. If the power generation is higher than people's power consumption, due to the difficult storage of power energy, Can lead to the waste of electric energy. Therefore, if there is an accurate prediction of future electricity consumption, the waste of electric energy can be greatly avoided, and the stability of the power system can also be maintained.
  In the above background, power load forecasting has always been a relatively hot scientific research topic; under the current rapid development of deep learning technology, a large number of deep learning neural network models have been applied to power load forecasting, and relatively good forecasting has also been achieved result. Power load forecasting tasks can be divided into single-feature forecasting and multi-feature forecasting. Single-feature forecasting uses power load data to predict power load data, and multi-feature forecasting uses factors that affect power load and power load data to predict power load. data.
  This paper mainly uses BP neural network, RNN, LSTM neural network, and CNN-LSTM neural network in deep learning to perform single-feature power load forecasting, and compares the models.

---

1. Analysis of single characteristic power load data

1.1. Data display

  The experimental data set for deep learning of single-feature electric load forecasting is as follows:
  specific data can be obtained in my course:
  [hands-on teaching] based on BP neural network single-feature electric load forecasting
  [hands-on teaching] based on RNN, LSTM neural network Single-feature electricity load forecasting
  [hands-on teaching] Single-feature electricity load forecasting based on CNN-LSTM neural network

  The data is the data of a certain area in 2014, and the sampling point of the data is every 15 minutes, so there are a total of 96 sampling points. Therefore, the above data has a total of 365 rows and 96 columns of data, each row corresponds to the power load data of one day, and each column corresponds to the power load data of one of the sampling points. Because it is a study of single-feature power load forecasting, there are no other features in the data except power load data.

1.2. Analyze the law in the power load

  As shown in the figure below, it is a display map of electric load data for a whole year in 2014 in a certain region.

  It can be seen from the figure that in February, the power consumption in this region was at the lowest level in the whole year. At the same time, the overall electricity consumption is at a relatively high level from June to September, which may be related to the weather. The hot weather during this period leads to relatively high electricity consumption.
  At the same time, carefully observe the power consumption in the picture, it seems to be in a state of ups and downs and repeated cycles. From the common sense of life, the power consumption has a strong correlation with people's work and rest. So let's take a look at what the weekly power load diagram looks like.

  As shown in the figure above, the weekly power load value of the area is displayed in the form of a graph; it can be seen from the figure that the power consumption from Monday to Friday on weekdays is higher than the point peak value used on Saturday and Sunday on rest days is high, which may indicate that a lot of electrical energy is being put into work. At the same time, the electricity load waveform in the figure is in a cyclic state, which obviously shows that there is a strong correlation between electricity consumption and people's work and rest; when people are resting, they maintain a very low electricity consumption state. When people work and live, the consumption of electric energy is in a very high state; and this state is constantly repeated.

---

2. Single-feature electric load forecasting based on BP neural network

2.1. BP neural network model applied to single-feature power load forecasting

  Single-feature electric load forecasting is essentially a regression task, that is, the output value of the final neural network is a continuous uncertain value.
  The data input in the neural network is called a feature, which is generally represented by X, and the output of the neural network is generally represented by y; at the same time, the regression prediction task is a supervised learning task, and the data has a corresponding label, which is the neural network. The correct answer is usually represented by Y. There is a certain error between the output y of the neural network and the label Y. Using this error, the backpropagation of the neural network can be performed. Using backpropagation, the error between the output y and Y of the neural network is continuously reduced until it is close to 0. .
  The above is that the single-feature load forecasting model continuously learns useful information from the data so that the final prediction result of the model is close to the real value. In the task of single-feature load forecasting, even if the power load data is input with feature X, it is also the real label Y of the data, and The input has and only has power load data. Compared with the multi-feature power load forecasting task, the input features at this time not only include power load data, but also may include factors that affect power load.
  Let's take a look at how the BP neural network uses power load data to perform power load forecasting tasks. The structure of the BP neural network model in this task is as follows:

  As shown in the figure above, it is a model diagram of the BP neural network for single-feature power load forecasting. It can be seen from the figure that the input of the BP neural network only contains the power load value. The neural network contains two hidden layers, and the calculation of the hidden layer finally outputs the output value of the neural network through the output layer. This value is the predicted value of the electrical load.

---

2.2. Data preprocessing and data set division

  The following is the preprocessing and division part of the single-feature electric load forecasting data set:

# 获取前80%作为数据的训练集80%到90%作为验证集
# 划分训练集和验证集
train = dataset.iloc[0:int(len(a)*0.8), [0]]
val = dataset.iloc[int(len(a)*0.8):int(len(a)*0.9), [0]]


# 进行数据归一化，将数据归一化到0-1之间
scaler = MinMaxScaler(feature_range=(0, 1))
train = scaler.fit_transform(train)
val = scaler.fit_transform(val)


"""
进行训练集数据特征和对应标签的划分，其中前面96个采样点中的负荷特征
来预测第97个点的电力负荷值。
"""

# 设置训练集的特征列表和对应标签列表
x_train = []
y_train = []


# 将前96个采样点的天气特征和电价特征和负荷特征作为训练特征添加到列表中
# 按照上述规律不断滑动取值
for i in np.arange(96, len(train)):
    x_train.append(train[i - 96:i, :])
    y_train.append(train[i])

# 将训练集由list格式变为array格式
x_train, y_train = np.array(x_train), np.array(y_train)
x_train, y_train = np.reshape(x_train, (x_train.shape[0], 96)), np.reshape(y_train, (y_train.shape[0], 1))

# print(x_train.shape)
# print(y_train.shape)


# 设置训练集的特征列表和对应标签列表
x_val = []
y_val = []


# 将前48个采样点的天气特征和电价特征和负荷特征作为训练特征添加到列表中
# 按照上述规律不断滑动取值
for i in np.arange(96, len(val)):
    x_val.append(val[i - 96:i, :])
    y_val.append(val[i])

# 将训练集由list格式变为array格式
x_val, y_val = np.array(x_val), np.array(y_val)
x_val, y_val = np.reshape(x_val, (x_val.shape[0], 96)), np.reshape(y_val, (y_val.shape[0], 1))

  First of all, normalize the acquired data. The numbers in the data tend to be relatively large, which makes the calculation of the time network slower. Therefore, the first step in data prediction is to perform normalized preprocessing on it. The features are normalized between 0-1. Since the experimental data is a sampling point of 15 minutes, that is, 96 sampling points a day, when setting the input and label of the neural network, set the first 1-96 sampling points of the data to predict the 97th point, 2 -97 sampling points to predict the 98th point, and keep rolling according to the above rules.
  At the same time, set the list of features and labels corresponding to the training set and verification set to install the corresponding data and labels, so that the subsequent neural network can use the data features to learn, and reverse the error calculated by using the forward propagation output value and the real label. Update the weight parameters of the neural network to the propagation, so that the neural network can continuously learn useful information from the data.

---

2.3. Model building

The following is the BP neural network model built using the deep learning framework:

# 利用keras搭建BP神经网络，该网络隐藏层一共有两层，神经元分别为10
model = Sequential()
model.add(Dense(10, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(1))

# 对模型进行编译，选用Adam优化器，学习率为0.01
model.compile(optimizer=keras.optimizers.Adam(0.01), loss='mean_squared_error')

# 将训练集和测试集放入网络进行训练，每批次送入的数据为512个数据，一共训练30轮，将测试集样本放入到神经网络中测试其验证集的loss值
history = model.fit(x_train, y_train, batch_size=512, epochs=30, validation_data=(x_val, y_val))

# 保存训练好的模型
model.save('BP_model.h5')

At this point, the BP neural network has been built, and the divided training set can be used to train the neural network.
The following is the training process of the BP power load network. From the figure below, we can see that the loss of the training set and verification set of the neural network is constantly decreasing. Obviously, the neural network model is constantly converging. Take a look at the final loss value comparison chart of the BP neural network as shown below: As shown in the

figure above, this figure records the loss values ​​of the training set and the verification set during the 30 rounds of training of the BP neural network. Obviously, after 20 rounds The neural network model has converged.

---

2.4. Model prediction

  From the project folder, you can see that the model file that has been saved and trained has been generated. Using the model parameter file, you can directly perform model inference on the characteristics of the input data. The generated model file is shown in the figure below: At this time, you can use the trained model

  to Test to test and denormalize the predicted values. The specific code is as follows:

# 导入训练好的模型权重文件
model = load_model("BP_model.h5")


# 测试集输入模型进行预测
predicted = model.predict(x_test)
# print(predicted.shape)
# print(test.shape)

# 将真实值标签进行反归一化操作，方便后续和真实值进行对比
real = scaler.inverse_transform(test[96:])

# 将模型预测出的值进行反归一化操作
prediction = scaler.inverse_transform(predicted)

  Compare the predicted value with the real label, and draw the comparison chart of the predicted value and the real value as follows.

  From the comparison in the above figure, it can be found that the BP neural network has a certain effect on power load forecasting, but it can be seen that there are certain errors between the peak and peak valley predictions and the actual values. Specifically, the evaluation index of the model can be used to evaluate the model.

# 计算模型的评价指标
R2 = r2_score(real, prediction)
MAE = mean_absolute_error(real, prediction)
RMSE = np.sqrt(mean_squared_error(real, prediction))
MAPE = np.mean(np.abs((real-prediction) / prediction))

# 打印模型的评价指标
print('R2:', R2)
print('MAE:', MAE)
print('RMSE:', RMSE)
print('MAPE:', MAPE)

  The specific evaluation index calculation is as follows:

  For more detailed explanations and complete codes, you can go to my course to obtain:
  [hands-on teaching] single-feature electricity load forecasting based on BP neural network
  [hands-on teaching] single-feature electricity consumption based on RNN and LSTM neural network Load forecasting
  [hands-on teaching] based on CNN-LSTM neural network single-feature electricity load forecasting

---

3. Single-feature electric load forecasting based on RNN and LSTM neural network

3.1. RNN and LSTM neural network models are applied to single-feature power load forecasting

In the article, the principle   of the RNN/LSTM neural network model has been explained in detail. If you don’t understand it, you can read this article in detail . Take a look at the course I recorded, which explains this model in detail.
  [hands-on teaching] Based on BP neural network single-feature electricity load forecasting
  [hands-on teaching] based on RNN, LSTM neural network single-feature electricity load forecasting
  [hands-on teaching] based on CNN-LSTM neural network single-feature electricity load forecasting
  Now we already know The model of the RNN neural network and the model of the LSTM neural network can be represented by the following diagram:

  As shown in the figure above, this figure is a comparison chart between the RNN neural network and the LSTM neural network. The upper figure is the structural diagram of the RNN neural network, and the lower figure is the structural diagram of the LSTM neural network. Obviously, the neural network unit is different. Besides, RNN neural network and LSTM neural network are basically the same. Then let's take the LSTM neural network as an example to see how the LSTM neural network performs single-feature power load forecasting tasks.

  As shown in the figure above, it shows how multi-featured power load data is input into the LSTM neural network; here it is important to emphasize that this figure cannot be viewed as a whole, but should be viewed from left to right, that is, in order. The data input for 3 time steps is plotted in the figure.
  First, input the power load data value at time t-1 to calculate the LSTM neural network unit, and the calculated values ​​are sent to two places respectively. The first one is used as the output of the hidden layer of the neural network to the next layer, and the second one is used as the output of the hidden layer of the neural network. As the input of the next time step; therefore, the data input at time t is the power load data at that time and the output result of the time step at the previous time, so the input at time t includes the information of the previous time. The output at time t has the same two outputs as at time t-1. The calculation at time t+1 is the same as the calculation at time t, so the calculation continues according to the above-mentioned rule until the calculation ends at the last set time.
  The data in this article is a sampling point every 15 minutes, that is, 96 points will be sampled a day. Since the power load data analyzed above changes periodically with people's work and rest, in the single-feature power load forecasting, the load data of the first 96 sampling points are taken as the input features of the model to predict the power of the 97th sampling point Load data; keep scrolling according to the law.
  Therefore, in this experiment, the LSTM neural network must continuously calculate 96 time steps until the final predicted value output. Here I want to emphasize that calculating 96 time steps does not mean that there are 96 calculation units mentioned above, but that the same unit repeats calculations 96 times, so such a neural network is also called a cyclic neural network. Similarly, the calculation process of RNN neural network and LSTM neural network is the same.

---

3.2. Data preprocessing and data set division

  The following is the preprocessing and division part of the power load forecasting data set:

# 进行数据归一化，将数据归一化到0-1之间
scaler = MinMaxScaler(feature_range=(0, 1))
train = scaler.fit_transform(train)
val = scaler.fit_transform(val)


"""
进行训练集数据特征和对应标签的划分，其中前面96个采样点中的负荷特征
来预测第97个点的电力负荷值。
"""

# 设置训练集的特征列表和对应标签列表
x_train = []
y_train = []

# 将前96个采样点的天气特征和电价特征和负荷特征作为训练特征添加到列表中
# 按照上述规律不断滑动取值
for i in np.arange(96, len(train)):
    x_train.append(train[i - 96:i, :])
    y_train.append(train[i])

# 将训练集由list格式变为array格式
x_train, y_train = np.array(x_train), np.array(y_train)

# 设置训练集的特征列表和对应标签列表
x_val = []
y_val = []

# 将前96个采样点的天气特征和电价特征和负荷特征作为训练特征添加到列表中
# 按照上述规律不断滑动取值
for i in np.arange(96, len(val)):
    x_val.append(val[i - 96:i, :])
    y_val.append(val[i])

# 将训练集由list格式变为array格式
x_val, y_val = np.array(x_val), np.array(y_val)

  The above is the data preprocessing of the LSTM neural network model and the division of the training set and the verification set. It is the same as the preprocessing of the previous BP neural network data set. After the data is obtained, the data must be normalized. This step While accelerating the update of the parameters of the neural network, it is also conducive to improving the prediction accuracy of the neural network.
  It should be emphasized here that the data format of the output network model of RNN and LSTM is [number of samples, time step, number of features]; the number of samples is easy to understand, that is, how many samples are there in the training set; the time step is the cyclic neural network How many calculations does the network need to cycle in total? For example, the experiment in this case uses the influence factors and load data of the previous 96 time steps as features to predict the data of the 97th point. Then the cycle neural network needs to cycle 96 times, each time Calculate the characteristic data of a time step; the characteristic data is also easy to understand, the characteristic in this experiment is the load, (here it should be noted that the electric load data is both a characteristic and a label). So let's take a look at the data dimensions of the final data set:

  Obviously, the training set has 27936 samples, the time step is 96, and the feature is 1; obviously the validation set has 3408 samples, the time step is 96, and the feature is 1. .

---

3.3. Model building

  The following is the RNN neural network model built using the deep learning framework:

# 利用keras搭建RNN神经网络，该网络隐藏层一共有两层，神经元分别为10
model = Sequential()
model.add(SimpleRNN(10, return_sequences=True, activation='relu'))
model.add(SimpleRNN(10, return_sequences=False, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(1))

# 对模型进行编译，选用Adam优化器，学习率为0.01
model.compile(optimizer=keras.optimizers.Adam(0.01), loss='mean_squared_error')

# 将训练集和测试集放入网络进行训练，每批次送入的数据为512个数据，一共训练30轮，将测试集样本放入到神经网络中测试其验证集的loss值
history = model.fit(x_train, y_train, batch_size=512, epochs=30, validation_data=(x_val, y_val))

# 保存训练好的模型
model.save('RNN_model.h5')

  The following is the LSTM neural network model built using the deep learning framework:

# 利用keras搭建LSTM神经网络，该网络隐藏层一共有两层，神经元分别为10
model = Sequential()
model.add(LSTM(10, return_sequences=True, activation='relu'))
model.add(LSTM(10, return_sequences=False, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(1))

# 对模型进行编译，选用Adam优化器，学习率为0.01
model.compile(optimizer=keras.optimizers.Adam(0.01), loss='mean_squared_error')

# 将训练集和测试集放入网络进行训练，每批次送入的数据为512个数据，一共训练30轮，将测试集样本放入到神经网络中测试其验证集的loss值
history = model.fit(x_train, y_train, batch_size=512, epochs=30, validation_data=(x_val, y_val))

# 保存训练好的模型
model.save('LSTM_model.h5')

  The following uses the LSTM neural network as an example to explain. The training process of the LSTM neural network is shown in the figure below. It can be seen that the loss value is continuously decreasing. After the model training is completed, the loss value of the model is as follows: Obviously, the network is close to convergence in about 5 rounds

  . For BP neural network, the convergence is faster.

---

3.4. Model prediction

  After the model training is completed, the trained model is saved, and the model parameter file can be used to directly perform model inference on the input data characteristics. The generated model parameters are as follows:

  At this time, the trained model can be used to test the test, and the predicted predicted value can be denormalized. The specific code is as follows:

model = load_model("LSTM_model.h5")

# 测试集输入模型进行预测
predicted = model.predict(x_test)
# print(predicted.shape)
# print(test.shape)

# 将真实值标签进行反归一化操作，方便后续和真实值进行对比
real = scaler.inverse_transform(test[96:])


# 将模型预测出的值进行反归一化操作
prediction = scaler.inverse_transform(predicted)

  Compare the predicted value with the real label, and draw the comparison chart of the predicted value and the real value as follows.

  From the comparison in the above figure, it can be found that the effect of LSTM neural network on power load forecasting is very good, and the model can be evaluated by using the evaluation index of the model.

# 计算模型的评价指标
R2 = r2_score(real, prediction)
MAE = mean_absolute_error(real, prediction)
RMSE = np.sqrt(mean_squared_error(real, prediction))
MAPE = np.mean(np.abs((real-prediction) / prediction))

# 打印模型的评价指标
print('R2:', R2)
print('MAE:', MAE)
print('RMSE:', RMSE)
print('MAPE:', MAPE)

  The specific evaluation index calculation is as follows

  More detailed explanation and complete code can be obtained in my course:
  [hands-on teaching] based on BP neural network single-feature electricity load forecasting
  [hands-on teaching] based on RNN, LSTM neural network single-feature electricity load Forecasting
  [hands-on teaching] based on CNN-LSTM neural network single-feature electricity load forecasting

---

4. Single-feature electric load forecasting based on CNN-LSTM neural network

4.1. CNN-LSTM neural network model applied to multi-feature power load forecasting

  Compared with a single neural network model, in some scenarios, the effect of a combined model is often better than that of a single model, because the combined model often has complementary advantages. The CNN-LSTM network model is the first combined neural network model explained in this series. It is not difficult to find out from the name of the neural network model that the neural network model is a combination of CNN neural network and LSTM neural network to learn and predict power load data. But here I want to emphasize that the CNN neural network used here is a one-dimensional convolutional neural network; and the convolutional neural network that everyone comes into contact with during the learning process of deep learning is basically a two-dimensional convolutional neural network, because compared to other As far as the neural network is concerned, the two-dimensional convolutional neural network is indeed better than other neural networks in the feature extraction of images; but this paper uses the combined neural network model to extract the features of the power load data. From the above analysis, we can Know that the electric load data is a typical time series model data. Therefore, one-dimensional convolutional neural network can be used to extract time series data. Before learning the one-dimensional convolutional neural network, it is recommended to learn the deep learning principle of the two-dimensional convolutional neural network ----- convolutional neural network . Here are related articles and corresponding video courses I wrote.
  [hands-on teaching] based on BP neural network single-feature electricity load forecasting
  [hands-on teaching] based on RNN, LSTM neural network single-feature electricity load forecasting
  [hands-on teaching] based on CNN-LSTM neural network single-feature electricity load forecasting

4.1.1. One-dimensional convolution operation

  Compared with the two-dimensional convolution operation, which performs convolution operations by sliding left, right, up and down on the entire feature map, the one-dimensional convolution operation only performs continuous sliding on the data feature H for convolution operations. Specifically as shown in the figure.

  As shown in the figure above, assuming that the dimension of the data at this time is H×W (of course, in the single-feature power load forecasting task, since the data input has only one feature of load, the dimension of the data at this time is often H×1), then this As long as the height of the convolution kernel is set, as shown in the figure, if the height of the convolution kernel is set to 3, then the dimension of the convolution kernel must be 3×W, because the one-dimensional convolution operation only performs one direction Sliding; of course, the convolution operation can also be performed in two-dimensional convolution operations such as setting the stride. Use the set convolution kernel to continuously slide on the data features, multiply the corresponding factors with the convolution kernel and the corresponding receptive field, and finally sum. Repeat the operation until you can no longer swipe down. The specific operation is shown in the figure above.
  Of course, the data for one-dimensional convolution is not necessarily single-channel data. The data may have multiple channels. At the same time, it is hoped to artificially set the channel of the output data after convolution operation. The specific calculation is shown in the figure below: As shown in the figure above

  , Suppose the dimension of the input data is H×W×C; then the number of channels of the corresponding convolution kernel must be C at this time, and the width of the convolution kernel of the one-dimensional convolution operation must be the same as the width of the feature map, then The width of the convolution kernel is also W; at the same time, set the height of the convolution kernel to FH, then the dimension of the convolution kernel is FH×W×C; if you want to set the number of channels of the output data, then you must set the convolution The number of kernels; assuming that the channel size of the data to be output is FN, then the number of convolution kernels must be set to FN. Therefore, the dimension of the final output data by calculation is OH×1×FN.

4.1.2. One-dimensional pooling operation

  (1) Maximum pooling
  Maximum pooling is to select the maximum value for the selected area, which is the same as the two-dimensional pooling operation, as shown in the following figure:

  (2) Average pooling
  Average pooling is to select the average value for the selected area, which is the same as the two-dimensional pooling operation, as shown in the following figure:

4.2. CNN-LSTM model applied to electric load forecasting with multiple influencing factors

  After the analysis of the above one-dimensional convolution operation and pooling operation, the combined neural network model of CNN-LSTM as shown in the figure below is built.

  It can be seen from the figure that the input of the neural network network is single-featured power load data, and the data is sampled every 15 minutes, so the sampling of a day is 96 time points, and the power load data is a typical time series data. The previous data The time data has a certain influence on the following data, so the data of the first 96 sampling points are input as the input of the neural network.
First, one-dimensional convolution operation is performed on the data. It can be seen from the above that the data after one-dimensional convolution operation is an N×1-dimensional data. In the figure, seven convolution kernels are used to perform one-dimensional convolution algorithm, so the obtained 7 N×1-dimensional data; that is, to obtain a data channel with 7 data channels and a length and width of N×1 data.
  Then perform a one-dimensional pooling operation on the convoluted data. Regardless of whether it is two-dimensional pooling or one-dimensional pooling, the number of data channels after the pooling operation remains unchanged. The length of the data after the one-dimensional pooling operation is smaller than the original (of course, it may remain the same or become larger, depending on the specific parameter values ​​​​of the pooling operation). The data after the pooling operation can also be subjected to one-dimensional convolution operation, but in order to facilitate the observation of the overall structure, the model in this paper draws a convolution and a pooling operation.
  Finally, the feature structure extracted by the convolutional neural network is input into the LSTM neural network, and the time series information is learned by the LSTM neural network. And at the end of the neural network structure, the fully-connected neural network layer is connected, and the fully-connected neural network predicts the previously extracted and learned features, and finally outputs the predicted value.

---

4.3. Data preprocessing and data set division

  The following is the preprocessing and division part of the power load forecasting data set:

# 进行数据归一化，将数据归一化到0-1之间
scaler = MinMaxScaler(feature_range=(0, 1))
train = scaler.fit_transform(train)
val = scaler.fit_transform(val)


"""
进行训练集数据特征和对应标签的划分，其中前面96个采样点中负荷特征
来预测第97个点的电力负荷值。
"""

# 设置训练集的特征列表和对应标签列表
x_train = []
y_train = []


# 将前96个采样点的天气特征和电价特征和负荷特征作为训练特征添加到列表中
# 按照上述规律不断滑动取值
for i in np.arange(96, len(train)):
    x_train.append(train[i - 96:i, :])
    y_train.append(train[i])

# 将训练集由list格式变为array格式
x_train, y_train = np.array(x_train), np.array(y_train)


# print(x_train.shape)
# print(y_train.shape)


# 设置训练集的特征列表和对应标签列表
x_val = []
y_val = []


# 将前48个采样点的天气特征和电价特征和负荷特征作为训练特征添加到列表中
# 按照上述规律不断滑动取值
for i in np.arange(96, len(val)):
    x_val.append(val[i - 96:i, :])
    y_val.append(val[i])

# 将训练集由list格式变为array格式
x_val, y_val = np.array(x_val), np.array(y_val)

---

4.4. Model building

  The following is the CNN-LSTM neural network model built using the deep learning framework:

# 利用keras搭建RNN神经网络，该网络隐藏层一共有两层，神经元分别为10
model = Sequential()
model.add(Conv1D(filters=32, kernel_size=3, strides=1, padding="same", activation="relu"))
model.add(MaxPooling1D(pool_size=2, strides=1, padding="same"))
model.add(Conv1D(filters=64, kernel_size=2, strides=1, padding="same", activation="relu"))
model.add(MaxPooling1D(pool_size=3, strides=1, padding="same"))
model.add(LSTM(10, return_sequences=True, activation='relu'))
model.add(LSTM(10, return_sequences=False, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(1))

# 对模型进行编译，选用Adam优化器，学习率为0.01
model.compile(optimizer=keras.optimizers.Adam(0.01), loss='mean_squared_error')

# 将训练集和测试集放入网络进行训练，每批次送入的数据为512个数据，一共训练30轮，将测试集样本放入到神经网络中测试其验证集的loss值
history = model.fit(x_train, y_train, batch_size=512, epochs=30, validation_data=(x_val, y_val))

# 保存训练好的模型
model.save('CNN_LSTM_model.h5')

  The figure below shows the loss value map of the CNN-LSTM neural network. It can be seen from the figure that the loss value of the neural network drops rapidly. It shows that the network converges very quickly.

---

4.5. Model prediction

  After the model training is completed, the trained model is saved, and the model parameter file can be used to directly perform model inference on the characteristics of the input data. The generated model parameters are as follows: At this time, the trained

  model can be used to test the test, and the predicted prediction Values ​​are denormalized. The specific code is as follows:

model = load_model("CNN_LSTM_model.h5")


# 测试集输入模型进行预测
predicted = model.predict(x_test)
# print(predicted.shape)
# print(test.shape)

# 将真实值标签进行反归一化操作，方便后续和真实值进行对比
real = scaler.inverse_transform(test[96:])


# 将模型预测出的值进行反归一化操作
prediction = scaler.inverse_transform(predicted)

  Compare the predicted value with the real label, and draw the comparison chart of the predicted value and the real value as follows.

  From the comparison in the above figure, it can be found that the CNN-LSTM neural network can still predict the power load. Specifically, the model can be evaluated by using the evaluation index of the model.
  The specific evaluation index codes are as follows:

# 计算模型的评价指标
R2 = r2_score(real, prediction)
MAE = mean_absolute_error(real, prediction)
RMSE = np.sqrt(mean_squared_error(real, prediction))
MAPE = np.mean(np.abs((real-prediction) / prediction))

# 打印模型的评价指标
print('R2:', R2)
print('MAE:', MAE)
print('RMSE:', RMSE)
print('MAPE:', MAPE)

  The specific evaluation index calculation is as follows:

  For more detailed explanations and complete codes, you can go to my course to obtain:
  [hands-on teaching] single-feature electricity load forecasting based on BP neural network
  [hands-on teaching] single-feature electricity consumption based on RNN and LSTM neural network Load forecasting
  [hands-on teaching] based on CNN-LSTM neural network single-feature electricity load forecasting

---

5. Summary of comparative analysis of single-feature power load forecasting models

  We have built BP, LSTM, and CNN-LSTM neural networks, and tested them with the test set, and calculated the index of the corresponding model's evaluation index. Next, we will use the value of such evaluation index to evaluate the above-built neural network. model for analysis. The specific evaluation index values ​​are shown in the table below:

Model / bid evaluation	R2	MAE	RMSE	MAPE
BP	0.976	186.134	231.039	0.0208
LSTM	0.995	80.457	108.091	0.011
CNN-LSTM	0.994	91.2	118.535	0.013

  As shown in the above table, it is the evaluation index of multi-feature electric load forecasting using BP, LSTM, and CNN-LSTM neural network in this paper; the evaluation indexes include R2, MAE, RMSE, and MAPE; R2 means that the value closer to 1 is better, and MAE , RMSE, and MAPE are the closer to 0, the better; from the table, it can be concluded that LSTM has the best effect, followed by CNN-LSTM, and BP has the worst effect. The analysis can draw the following conclusions:
  1 , The amount of data in this article is relatively sufficient, so the final three algorithm models are actually very good in terms of effect. It can predict the electric load value very well.
  2. Judging from a large number of papers, it is finally proved that the effect of CNN-LSTM is better than that of LSTM, but the effect obtained in this paper is that LSTM has a better effect. The reasons are as follows: (1) LSTM and
  CNN- The model parameters of LSTM have not been tuned, so it is not necessarily the optimal model.
  (2) This article does not carry out statistics under a large number of experiments, but only the experimental results obtained from one or two experiments, which may be accidental.
  (3) The data in this article is relatively sufficient. Whether the superiority of the CNN-LSTM model can be shown when the data is relatively small.
  (4) Although the effect of the CNN-LSTM model is worse than that of the LSTM, whether it is from the model's prediction map or the model's evaluation index, the direct difference between the two models is actually very small; therefore, whether the effect of the LSTM model is not good In some cases, the effect of using CNN-LSTM will be better.

---

6. Subsequent update plan of power load forecasting model

  The following articles and teaching videos related to the power load forecasting model of related algorithms are expected to be updated by the end of 23. If you want to continue to learn power load forecasting, or need to complete the design, you can follow me (or +q) 1343077375, and at the same time Article collection, I will continue to update related articles and teaching videos.
  6.1. Single-feature electricity load forecasting based on BP, LSTM, and CNN-LSTM neural network algorithms (end)
  6.2. Multi-feature electricity load forecasting based on BP, RNN, LSTM, and CNN-LSTM algorithms (end)
  6.3. Bayesian-based BP, RNN, LSTM, and CNN-LSTM algorithms for power load forecasting
  optimized by Sri Lanka 6.4, BP, RNN, LSTM, and CNN-LSTM algorithms for power load forecasting based on particle swarm optimization
  6.5, BP, RNN, and LSTM based on ant colony algorithm optimization , CNN-LSTM algorithm power load forecasting
  6.6, BP, RNN, LSTM, CNN-LSTM algorithm power load forecasting based on genetic algorithm
  6.7, BP, RNN, LSTM, CNN-LSTM algorithm power load forecasting based on wolf pack algorithm
  6.8 , BP, RNN, LSTM, CNN-LSTM algorithm-based power load forecasting based on wolf pack algorithm
  6.9, BP, RNN, LSTM, CNN-LSTM algorithm-based power load forecasting based on attention mechanism
  6.10, power load forecasting based on transformer algorithm