Recently, I have learned about time series. Next is a case study of hybrid neural networks applied to time series.
Article: " Hybrid Neural Networks for Learning the Trend in Time Series "
TreNet: A novel end-to-end hybrid neural network for learning local and global background features to predict trends in time series.
Use long-short memory recurrent neural network (RNN) LSTM to capture the long-range dependence in historical trend sequences.
CNN is usually used to extract distinctive features from the original data, and extract features from the original time series data for activity / action recognition. The
feature fusion layer will learn to predict the joint representation of trends.
TreNet proved the effectiveness of TreNet through the cascade of CNN, LSTM, CNN and LSTM, the method based on hidden Markov model and various kernel-based baselines based on real data sets.
Time series are a series of chronological data points that are being generated in a wide range of fields, such as medical and biological experiment observations, daily fluctuations in the stock market, power consumption records, performance monitoring of data centers, and so on.
Basic representation:
Sequence of data points:
Actual trend
on X: a linear function representing a certain subsequence, corresponding to the trend. lk and sk respectively represent duration and trend slope k, continuous values, and T do not overlap each other.
Define each historical trend of local data as a set of data points of size w:
Objective: To propose a neural network-based method to learn functions to predict subsequent trends.
The point is here:
TreNet's idea is to combine CNN and LSTM to take advantage of their representation capabilities in different aspects of the data (ie, L and T) and learn joint features for trend prediction. Predict and calculate on time T and slope L respectively.
Trend and R (T) and C (L) related functions.
R (T): Train the LSTM on the sequence L to capture the dependence in the historical trend changes.
C (L): Corresponds to the local features extracted by CNN from the local dataset in L.
Feature fusion layers merge features used to predict subsequent trends. Finally, trend prediction is implemented by a function, which corresponds to feature fusion and output layer.
Learn the dependencies in the historical trend sequence First explain the
formula
: Represents the storage of the jth neuron in the LSTM layer in step k. For output.
It is the output gate of LSTM neuron.
Is the concatenation of the duration of the trend k and the slope, and hk-1 and ck are the vectorization of, respectively.
σ is a logistic sigmoid function, you can Baidu if you don't understand it.
Update in the following manner, remove some existing records, and add new records:
forgetting control (forgetting to existing) and input control (getting new):
Vf, Vi are diagonal matrices.
It is the ReLU activation function of element leakage, Wo and bo are the weight and deviation of the output layer.
W and b represent the weight and deviation parameters.
λ is the hyperparameter of the regularization term. The cost function is differentiable, and the architecture of TreNet allows the gradient of the loss function to be propagated back to the LSTM and CNN parts.
Experimental analysis:
TreNet is compared with CNN, LSTM, ConvNet + LAT (CLSTM), SVR, pattern-based HMM, Naïve.
The evaluation parameters are: RMSE, root mean square error
Data sets are: Power Consumption (PC), Gas Sensor (GasSensor), Stock Transaction (Stock).
The comparison results are posted directly below:
RMSE predicts the trend duration and slope of each data set.
TreNet visualizes trend predictions for examples from PC, Stock and GasSensor datasets. The yellow dots represent time series, and the purple line in each graph represents the related historical trend series.
TreNet shows better prediction characteristics and stability.
Finally finished writing, ready to relax on the weekend.
