informer custom data set

Table of contents

informer-related

Model dataset partition

Model parameters

Run through custom data sets

Visualization of prediction results

informer-related

Paper: https://arxiv.org/abs/2012.07436
Thanks to the author of the paper for his contribution to AI science, ordinary people like me can have access to such nice articles. The author's github: GitHub - zhouhaoyi/Informer2020: The GitHub repository for the paper "Informer" accepted by AAAI 2021.

Model dataset partition

Suppose the data dimension is 317 days of data, the sliding window is 120 days to predict the next 20 days of data, num_train = 221, num_test = 63, num_vali = 33, border1s = [0, 101, 134], border2s = [221, 254, 317 ],

Training set data: 0-222 days, 120-day sliding window to predict the next 30 days, number of training samples: 222-120-30+1=72

Validation set data: 102-254 days, validation set sample number: 254-101-120-30+1=4

Test set data: 134 days-317 days, test set sample number: 317-134-120-30+1=34

The code part of the source code is as follows:

   # init
   assert flag in ['train', 'test', 'val']
   type_map = {'train':0, 'val':1, 'test':2}
   self.set_type = type_map[flag]

#自定义数据划分训练集、验证集、测试集部分
num_train = int(len(df_raw)*0.7)
num_test = int(len(df_raw)*0.2)
num_vali = len(df_raw) - num_train - num_test
border1s = [0, num_train-self.seq_len, len(df_raw)-num_test-self.seq_len]
border2s = [num_train, num_train+num_vali, len(df_raw)]
border1 = border1s[self.set_type]
border2 = border2s[self.set_type]

Model parameters

The overall framework of the model proposed in the paper is shown in the figure below. It can be seen that the proposed Informer model still preserves the Encoder-Decoder architecture:

model: can be set to informer, informersack, informerlight (to be determined);

data: data set name

root_path: the root path of the data file (default is ./data/ETT/)

data_path: data file name (default is ETTh1.csv)

features: prediction task (default is M). This can be set to M, S, MS (M: multivariate forecasting multivariate, S: univariate forecasting univariate, MS: multivariate forecasting univariate)

target: target target feature in S or MS task (default is OT)

freq: freq freq for temporal feature encoding (default h). This can be set to s, t, h, d, b, w, m (s: seconds, t: minutes, h: hours, d: daily, b: weekdays, w: weekly, m: monthly).

checkpoints: the location where the model is saved (default is ./checkpoints/)

seq_len: Informer encoder input sequence length (96 by default)

label_len: Informer decoder start label length (default is 48)

pred_len: predicted sequence length (24 by default)

enc_in: Encoder input size (default 7)

dec_in: decoder input size (default 7)

c_out: output size (default 7)

d_model: the size of the model (default is 512)

n_heads: number of heads (default 8)

e_layers: number of encoder layers (default 2)

d_layers: number of decoder layers (default 1)

s_layers: number of stacked encoder layers (default 3,2,1)

d_ff: the number of neurons in the fully connected layer (default is 2048)

factor: number of sampling factors (default is 5)

padding: 1D convolution kernel

distil: whether sequence length decay is required

dropout: neural network regularization operation

attn: attention calculation method

embed: time feature encoding method

activation: activation function

output_attention: whether to output attention

do_predict: whether to predict

mix: Whether to use mixed attention in the generative decoder, using this parameter means not to use mixed attention (default is True)

cols: some cols in the data file as input

itr: number of trials (default 2)

train_epochs: training epochs (default 6)

batch_size: batch size of training input data (default is 32)

patience: early stopping strategy (default is 3)

learning_rate: optimizer learning rate (default 0.0001)

loss: loss calculation method

lradj: learning rate decay parameter

use_amp: whether to use automatic mixed precision training

inverse: Whether to invert the output result

Run through custom data sets

After adding the custom data sequence dataset folder to the data folder, go to the code to modify the following places:
1. If the field name of the time column of the custom dataset is "date";

2. In main_informer, the points that need to be modified and paid attention to are as follows:

#想要获得最终预测的话这里应该设置为True；否则将是获得一个标准化的预测。
parser.add_argument('--inverse', action='store_true', help='inverse output data', default=True)

data_parser = {
    'ETTh1':{'data':'ETTh1.csv','T':'OT','M':[7,7,7],'S':[1,1,1],'MS':[7,7,1]},
    'ETTh2':{'data':'ETTh2.csv','T':'OT','M':[7,7,7],'S':[1,1,1],'MS':[7,7,1]},
    'ETTm1':{'data':'ETTm1.csv','T':'OT','M':[7,7,7],'S':[1,1,1],'MS':[7,7,1]},
    'ETTm2':{'data':'ETTm2.csv','T':'OT','M':[7,7,7],'S':[1,1,1],'MS':[7,7,1]},
    'WTH':{'data':'WTH.csv','T':'WetBulbCelsius','M':[12,12,12],'S':[1,1,1],'MS':[12,12,1]},
    'ECL':{'data':'ECL.csv','T':'MT_320','M':[321,321,321],'S':[1,1,1],'MS':[321,321,1]},
    'Solar':{'data':'solar_AL.csv','T':'POWER_136','M':[137,137,137],'S':[1,1,1],'MS':[137,137,1]},
   'custom':{'data':'train.csv','T':'flow','M':[5,5,5],'S':[1,1,1],'MS':[5,5,1]}
}
#'M':[5,5,5],其中5表示输入数据特征有5个

3. Here in the exp_informer file, the points that need to be modified and paid attention to are as follows:

Visualization of prediction results

1. After running the code without changing any parameters, two subfolders will be generated in the project folder: the checkpoints folder contains the trained model, the suffix is .pth, and the model file contains the complete model architecture With the weights of each layer, the model can be loaded through the torch.load function
2. The results folder contains three files: metrics.npy, pred.npy, and true.npy. pred.npy represents the predicted value of the model, and true.npy represents the real value of the sequence
3. Compare the pred.npy and true.npy files to observe the model effect

setting = 'informer_ETT_ftMS_sl120_ll90_pl30_dm256_nh8_el2_dl1_df2048_atprob_fc5_ebtimeF_dtTrue_mxTrue_test_0'
exp = Exp_Informer(args)
exp.predict(setting, True)

preds = np.load('D:Informer2020-main/results/' + setting + '/pred.npy')
trues = np.load('D:Informer2020-main/results/' + setting + '/true.npy')

print(trues.shape)
print(preds.shape)

plt.figure()
plt.plot(trues[0,:,-1].reshape(-1),label='GroundTruth')
plt.plot(preds[0,:,-1].reshape(-1),label='Prediction')
plt.legend()
plt.show()

4. Prediction result code

'--do_predict'Change the parameters before the pycharm model training , so that after the code runs, there will be an extra file in the folder 'store_true', which is the sequence value predicted by the model;'store_false'resultsreal_prediction.npy

#预测结果30天
setting = 'informer_ETT_ftMS_sl120_ll90_pl30_dm256_nh8_el2_dl1_df2048_atprob_fc5_ebtimeF_dtTrue_mxTrue_test_0'

preds = np.load('D:Informer2020-main/results/' + setting + '/real_prediction.npy')

print(preds .shape)