Machine learning and deep learning have been increasingly used in time series forecasting. Classical forecasting methods like ARIMA or exponential smoothing are being replaced by machine learning regression algorithms like XGBoost, Gaussian process or deep learning.
Despite the increasing complexity of timing models, there are doubts about the performance of timing models. Studies have shown that complex timing models are not necessarily more effective than timing decomposition models (Makridakis, 2018).
technology upgrade
Technology must learn to share and communicate, and it is not recommended to work behind closed doors.
Good articles are inseparable from the sharing and recommendation of fans, dry data, data sharing, data, and technical exchange improvement, all of which can be obtained by adding the communication group. The group has more than 2,000 friends. The best way to add notes is: source + interest directions, making it easy to find like-minded friends.
Method ①, Add WeChat account: dkl88191, Remarks: from CSDN
Method ②, WeChat search official account: Python learning and data mining, background reply: add group
Why is time series forecasting difficult?
Time series are values ordered in time, but time series forecasting is very challenging. From the perspective of model difficulty and accuracy, time series models are more difficult than conventional regression and classification tasks.
Reason 1: The sequence is non-stationary
Stationarity is a core concept of time series, a time series is stationary if its trend (e.g. average) does not change over time. Many existing methods assume that the time series is stationary, but trend or seasonality breaks the stationarity.
Reason 2: Reliance on external data
In addition to the time factor, time series often have additional dependencies. A common example is spatiotemporal data, where each observation is related in two dimensions, so the data has a lag for itself (time dependence) and a lag for nearby locations (spatial dependence).
Reason 3: Noise and missing values
The real world is plagued by noise and missing values, and equipment failures can produce noise and missing values. Data loss due to sensor failure, or interference, can cause data noise.
Reason 4: Limited sample size
Time series often contain only a small number of observations, and there may not be enough data to build an adequate model. The frequency of data collection affects the sample size and also encounters the data cold start problem.
Sample Size and Model Accuracy
Time series models are often unable to make perfect predictions, which may be related to the sample size of time series data. Models with large sizes tend to perform better than models with fewer parameters when using larger training sets. When the length of the time series is less than 1000, the deep model is often not better than the time series classification model.
The relationship between model accuracy and the number of samples is compared below. Five classic methods (ARIMA, ETS, TBATS, Theta and Naive) and five machine learning methods (Gaussian process, M5, LASSO, random forest and MARS) are tried here. The forecasting task is to predict the next value of a time series.
The results are shown in the figure below, with the axis representing the training sample size, the amount of data used to fit the predictive model. The axes represent the average error of each model across all time series, calculated using cross-validation.
The base method shows better performance when only a few observations are available. However, machine learning methods outperform classical methods as the sample size increases.
Further conclusions can be drawn:
-
Machine learning methods possess strong predictive power provided they have sufficiently large training datasets;
-
Classical methods such as ARIMA or exponential smoothing are recommended to be preferred when only a small number of observations are available;
-
Combining classical methods such as exponential smoothing with machine learning can improve forecast accuracy.
Time Series Multi-step Forecasting
Most forecasting problems are formulated as one-step forecasts, predicting the next value in a series based on recent events. Time series multi-step forecasting needs to predict multiple values in the future. Forecasting many steps in advance has important practical advantages, and multi-step forecasting reduces long-term uncertainty. But as the model tries to predict further into the future, the error of the model also gradually increases.
Method 1: Recursive Prediction
The simplest approach to multi-step forecasting is the recursive form, where a single model is trained to make a single-step forecast, and then the model and its previous predictions are used as input to obtain subsequent outputs.
from sklearn.linear_model import LinearRegression
# using a linear regression for simplicity. any regression will do.
recursive = LinearRegression()
# training it to predict the next value of the series (t+1)
recursive.fit(X_tr, Y_tr['t+1'])
# setting up the prediction data structure
predictions = pd.DataFrame(np.zeros(Y_ts.shape), columns=Y_ts.columns)
# making predictions for t+1
yh = recursive.predict(X_ts)
predictions['t+1'] = yh
# iterating the model with its own predictions for multi-step forecasting
X_ts_aux = X_ts.copy()
for i in range(2, Y_tr.shape[1] + 1):
X_ts_aux.iloc[:, :-1] = X_ts_aux.iloc[:, 1:].values
X_ts_aux['t-0'] = yh
yh = recursive.predict(X_ts_aux)
predictions[f't+{
i}'] = yh
The corresponding implementation of the above code logic can also be found in sktime: https://www.sktime.org/en/stable/api_reference/auto_generated/sktime.forecasting.compose.RecursiveTimeSeriesRegressionForecaster.html
Recursive methods require only one model for the entire forecast horizon and do not require prior determination of the forecast horizon.
However, this method uses its own forecast as input, which leads to gradual accumulation of errors and poor predictive performance for long-term forecasts.
Method 2: Multi-objective regression
Multi-objective regression builds a model for each predicted outcome, as follows for a use case:
from sklearn.multioutput import MultiOutputRegressor
direct = MultiOutputRegressor(LinearRegression())
direct.fit(X_tr, Y_tr)
direct.predict(X_ts)
scikit-learn MultiOutputRegressorreplicates a learning algorithm for each target variable. In this case the forecast method is LinearRegression.
This method avoids error propagation in a recursive manner, but multi-objective prediction requires more computing resources. In addition, multi-objective prediction assumes that each point is independent, which violates the characteristics of time series data.
Approach 3: Recursive Multi-Objective Regression
Recursive multi-objective regression combines the ideas of multi-objective and recursion. Build a model for each point. But at each step the input data is increased with the previous model's predictions.
from sklearn.multioutput import RegressorChain
dirrec = RegressorChain(LinearRegression())
dirrec.fit(X_tr, Y_tr)
dirrec.predict(X_ts)
This approach is known in the machine learning literature as chaining. scikit-learn provides an implementation for this through the RegressorChain class.
references
Makridakis, Spyros, Evangelos Spiliotis, and Vassilios Assimakopoulos. “Statistical and Machine Learning forecasting methods: Concerns and ways forward.” PloS one 13.3 (2018): e0194889.