Today I will introduce a newly published multivariate time series forecasting model SCNN this week. The core of this article is to use the idea of factor decomposition to modularize the multivariate time series forecasting problem, and benefit from the decomposition and modular modeling methods to realize the interpretable modeling of multivariate time series forecasting.
论文标题：Learning Structured Components: Towards Modular and Interpretable Multivariate Time Series Forecasting
Download link : https://arxiv.org/pdf/2305.13036.pdf
1
Decomposition-based modeling ideas
In time series forecasting, the idea of modeling based on decomposition is very common. Generally, the time series is decomposed into factors such as trend items and seasonal items, and it is easier to model each factor independently than directly modeling complex mixed sequences. The core idea of this paper is also decomposition, which decomposes the multivariate time series into four factors, namely growth cycle item, short cycle item, seasonal item, and inter-sequence correlation item, to model respectively. In this paper, it is assumed that the generation of the entire multivariate time series is derived from the following four equations, where Z0 is the representation of the original multivariate sequence, which can be disassembled into the above four modules, each module is defined by a scale factor and a location factor:
The following question is how to extract the scale and location of each module. The method in this article is as follows:
Long-period items - use a larger sliding window to calculate the mean and variance within the window as scale and factor;
Short-period item - the calculation method is the same as the long-period item, but a smaller sliding window will be used to prevent short-term information from being smoothed out;
Seasonal item - can be extracted by Fourier transform, a simplified assumption is added in the article, the length of the seasonal cycle is constant, so the window statistics method is directly used;
Sequence correlation item - Multivariate sequences need to consider the relationship between each sequence more than unit sequences. In this article, attention calculates the correlation score of two sequences, and uses this score to calculate your scale and factor.
2
Main Model Structure
Based on the above four decomposition modules, the main structure of the model is shown in the figure below, including two parts: Encoder and Decoder. The Encoder encodes the information of the 4 modules, and the Decoder extrapolates and predicts the information of the 4 modules, and produces a prediction structure.
After extracting the corresponding representation and factors such as scale and location through the four modules, the information of these four modules is spliced together, corresponding to Zn and Hn in the figure. The extraction process of each module is based on the multivariate time series generation assumption in the first section. For the original sequence, the scale and location of the long-period module are extracted first, and the representation after removing the long-period information is input to the next component to extract the season module. Extraction is carried out in the order of analogy. The remaining sequence after the extraction of the 4 modules is considered as the residual part.
In the Fusion layer, 1-dimensional convolution is used to fuse the information of each module.
In the forecasting process of each module, different forecasting methods are adopted according to the characteristics of the four modules. For the long-term module and seasonal module, there will be little change in the future, so the scale and location factor learned by the long-term module and seasonal module can be directly used for prediction. For example, the future value of the long-term module can be the result corresponding to the last value of the observation sequence; the seasonal module can be shifted according to the period window length set above (the first two lines in the figure below).
For the residual items, short-term items, and serial correlation items, their changes fluctuate greatly, and they are not as stable and easy to predict as the long-term modules and seasonal modules mentioned above. The method used in this paper is to learn a DNN network, map the representation vector, scale, location and other information of the corresponding module in a historical window, and predict the corresponding information in the future:
After obtaining the future prediction results of each module, one-dimensional convolution is still used to fuse the features of each module to obtain the final prediction result.
3
loss function
The loss function in this article adopts maximum likelihood estimation, predicts the mean and variance at each moment, and the variance is followed by a softplus to ensure that the value is positive, and the maximum likelihood loss is used for optimization:
Because at the bottom of the model, the four main modules may be decomposed uncleanly, resulting in the residual part retaining part of the information that has not been decomposed by the four modules, direct learning will cause the model to give the residual part a higher weight, affecting the model effect . In order to solve this problem, another branch is adopted in the paper, which completely masks the residual part and also produces a prediction result. The prediction results of the two branches are weighted and summed for optimization.
4
Experimental effect
This paper compares the effects of various multivariate time series forecasting models on multiple data sets, including unit sequence models and spatiotemporal forecasting models. From the effect point of view, the effect of the method proposed in this paper has improved significantly, basically above 5%.
From the case, this article has also conducted an analysis, and the model in this article can achieve a relatively stable and accurate prediction effect.
推荐阅读：
我的2022届互联网校招分享
我的2021总结
浅谈算法岗和开发岗的区别
互联网校招研发薪资汇总
2022届互联网求职现状，金9银10快变成铜9铁10！！
公众号：AI蜗牛车
保持谦逊、保持自律、保持进步
发送【蜗牛】获取一份《手把手AI项目》（AI蜗牛车著）
发送【1222】获取一份不错的leetcode刷题笔记
发送【AI四大名著】获取四本经典AI电子书