Detailed interpretation of multi-feature LSTM time series prediction code based on pytorch (with complete code)

LSTM Time Series Forecasting

Everyone must be familiar with the concept of LSTM neural network, so this article does not involve the interpretation of the concept of LSTM, but only explains how to use pytorch to use LSTM for time series prediction, and restore the entire process implemented using code.

Data Acquisition and Preprocessing

First, preview the data set used in this experiment. The data set has three characteristics, and the compressor outlet temperature in the last column is used as a label prediction (this data set is collected by me on git)

Define a function for reading xls files, where the data.iloc() function slices the data in the dataframe and returns the data and labels

# 文件读取
def get_Data(data_path):

    data=pd.read_excel(data_path)
    data=data.iloc[:,:3]  # 以三个特征作为数据
    label=data.iloc[:,2:] # 取最后一个特征作为标签
    print(data.head())
    print(label.head())
    return data,label

Use the normalization function in the preprocessing module in sklearn to normalize the data, where the data=data.values ​​function is to convert the data in the dataframe from the pd format to an np array, delete the axis label, and the fit_transform function is fit() The combination with transform() is to combine fit and transform, the result in one step, and finally return data, label and normalized label value

# 数据预处理
def normalization(data,label):

    mm_x=MinMaxScaler() # 导入sklearn的预处理容器
    mm_y=MinMaxScaler()
    data=data.values    # 将pd的系列格式转换为np的数组格式
    label=label.values
    data=mm_x.fit_transform(data) # 对数据和标签进行归一化等处理
    label=mm_y.fit_transform(label)
    return data,label,mm_y

After we normalize the data, the data is in np array format, and we need to convert it into a vector format and store it in a list. Therefore, first create two empty lists, create a for loop, and finally press the preprocessed data x.size(0),seq_length,features) are output to the list. Among them, seq_length represents the time step, x.size(0) represents the first dimension of the data, and features represents the number of features of the data. Print the dimensions of x,y and return x,y.

# 时间向量转换
def split_windows(data,seq_length):

    x=[]
    y=[]
    for i in range(len(data)-seq_length-1): # range的范围需要减去时间步长和1
        _x=data[i:(i+seq_length),:]
        _y=data[i+seq_length,-1]
        x.append(_x)
        y.append(_y)
    x,y=np.array(x),np.array(y)
    print('x.shape,y.shape=\n',x.shape,y.shape)
    return x,y

After the data and labels are prepared, the data can be separated, and the data can be separated into a training set and a test set. Define the split_data() function, where split_ratio is the set test set ratio. The ratio of the training set to the test set set in this experiment is 9:1, that is, split_ratio=0.1. Pack the separated data into Variable and package them separately, and convert the array into tensor format to obtain the test set and training set. Note that you must use the Variable function to encapsulate the data set, otherwise the subsequent torch iterations are not supported.

# 数据分离
def split_data(x,y,split_ratio):

    train_size=int(len(y)*split_ratio)
    test_size=len(y)-train_size

    x_data=Variable(torch.Tensor(np.array(x)))
    y_data=Variable(torch.Tensor(np.array(y)))

    x_train=Variable(torch.Tensor(np.array(x[0:train_size])))
    y_train=Variable(torch.Tensor(np.array(y[0:train_size])))
    y_test=Variable(torch.Tensor(np.array(y[train_size:len(y)])))
    x_test=Variable(torch.Tensor(np.array(x[train_size:len(x)])))

    print('x_data.shape,y_data.shape,x_train.shape,y_train.shape,x_test.shape,y_test.shape:\n{}{}{}{}{}{}'
    .format(x_data.shape,y_data.shape,x_train.shape,y_train.shape,x_test.shape,y_test.shape))

    return x_data,y_data,x_train,y_train,x_test,y_test

Load the packaged training set and test set into the iterable object torch.utils.data.DataLoader supported by torch. num_epochs is the calculated number of iterations, and return train_loader, test_loader, num_epochs. In this way, the data set is preprocessed , the model can be constructed.

# 数据装入
def data_generator(x_train,y_train,x_test,y_test,n_iters,batch_size):

    num_epochs=n_iters/(len(x_train)/batch_size) # n_iters代表一次迭代
    num_epochs=int(num_epochs)
    train_dataset=Data.TensorDataset(x_train,y_train)
    test_dataset=Data.TensorDataset(x_test,y_test)
    train_loader=torch.utils.data.DataLoader(dataset=train_dataset,batch_size=batch_size,shuffle=False,drop_last=True) # 加载数据集,使数据集可迭代
    test_loader=torch.utils.data.DataLoader(dataset=test_dataset,batch_size=batch_size,shuffle=False,drop_last=True)

    return train_loader,test_loader,num_epochs

model building

Using torch to build a model is nothing more than defining a class, defining a model instance and a forward propagation function in this class, it's that simple, let's take a look.

# 定义一个类
class Net(nn.Module):
    def __init__(self,input_size,hidden_size,num_layers,output_size,batch_size,seq_length) -> None:
        super(Net,self).__init__()
        self.input_size=input_size
        self.hidden_size=hidden_size
        self.num_layers=num_layers
        self.output_size=output_size
        self.batch_size=batch_size
        self.seq_length=seq_length
        self.num_directions=1 # 单向LSTM

        self.lstm=nn.LSTM(input_size=input_size,hidden_size=hidden_size,num_layers=num_layers,batch_first=True) # LSTM层
        self.fc=nn.Linear(hidden_size,output_size) # 全连接层

    def forward(self,x):
        # e.g.  x(10,3,100) 三个句子，十个单词，一百维的向量,nn.LSTM(input_size=100,hidden_size=20,num_layers=4)
        # out.shape=(10,3,20) h/c.shape=(4,b,20)
        batch_size, seq_len = x.size()[0], x.size()[1]    # x.shape=(604,3,3)
        h_0 = torch.randn(self.num_directions * self.num_layers, x.size(0), self.hidden_size)
        c_0 = torch.randn(self.num_directions * self.num_layers, x.size(0), self.hidden_size)
        # output(batch_size, seq_len, num_directions * hidden_size)
        output, _ = self.lstm(x, (h_0, c_0)) # output(5, 30, 64)
        pred = self.fc(output)  # (5, 30, 1)
        pred = pred[:, -1, :]  # (5, 1)
        return pred

First define an instance, which includes the required parameters input_size, hidden_size, num_layers, output_size, batch_size, seq_length. Setting self.num_directions to 1 means that this is a single-item LSTM, and then add an lstm layer and a fully connected layer fc, the input dimension of the lstm layer is (input_size=input_size, hidden_size=hidden_size, num_layers=num_layers), set, batch_first=True means shape=(batch_size, seq_size, hidden_size), the parameters of the fc layer are (hidden_size, output_size), return pred

training and testing

Train the model, initialize i, (batch_x, batch_y), set the train_loader as an enumeration type, optimizer.zero_grad() means to clear the gradient accumulation during each propagation, if optimizer.zero_grad() is not declared in torch, it will always accumulate Calculate the gradient and set the loss to be printed every 100 inputs

# train
iter=0
for epochs in range(num_epochs):
  for i,(batch_x, batch_y) in enumerate (train_loader):
    outputs = moudle(batch_x)
    optimizer.zero_grad()   # 将每次传播时的梯度累积清除
    # print(outputs.shape, batch_y.shape)
    loss = criterion(outputs,batch_y) # 计算损失
    loss.backward() # 反向传播
    optimizer.step()
    iter+=1
    if iter % 100 == 0:
      print("iter: %d, loss: %1.5f" % (iter, loss.item()))

The last few losses are as follows

iter: 2400, loss: 0.00331
iter: 2500, loss: 0.00039
...
iter: 4400, loss: 0.00332
iter: 4500, loss: 0.00022
iter: 4600, loss: 0.00380
iter: 4700, loss: 0.00032

Draw the MAE/RMSE of the final training set and test set to get the final result.

def result(x_data, y_data):
  moudle.eval()
  train_predict = moudle(x_data)

  data_predict = train_predict.data.numpy()
  y_data_plot = y_data.data.numpy()
  y_data_plot = np.reshape(y_data_plot, (-1,1))  
  data_predict = mm_y.inverse_transform(data_predict)
  y_data_plot = mm_y.inverse_transform(y_data_plot)

  plt.plot(y_data_plot)
  plt.plot(data_predict)
  plt.legend(('real', 'predict'),fontsize='15')
  plt.show()

  print('MAE/RMSE')
  print(mean_absolute_error(y_data_plot, data_predict))
  print(np.sqrt(mean_squared_error(y_data_plot, data_predict) ))

result(x_data, y_data)
result(x_test,y_test)

Final result: training set: MAE/RMSE: 35.114613\75.8706
test set: MAE/RMSE: 213.30313\213.31061
This article is only to demonstrate the usage of pytorch to build lstm, the prediction result is not very accurate, such as dropout is not added, for reference only.
See my github for the complete code: https://github.com/Tuniverj/Pytorch-lstm-forecast