Author: Song Tongtong
I believe many people will feel the ups and downs of the stock market data curious, they particularly want to know what will be the future trend, recently I saw an article made a preliminary forecast of the stock market with LSTM, here for everyone to share.
1. LSTM neurons
First we have to understand what LSTM neurons:
a fundamental problem long plagued traditional neural network architecture that interprets the input sequence "information" and "context" interdependent. Here the "information" may be the previous sentence words to allow the context of what is possible to predict a word or it can be a time sequence information to allow the context to make predictions based on the time sequence elements.
In short, the traditional neural network independent data every time the input vector, and there is no concept of memory needed to help them deal with memory tasks.
Early attempts to solve this problem is to use a simple network of neurons in the feedback type methods , wherein the output is fed back to the input to provide context entered last seen. These are called recurrent neural network (RNN). Although these RNN work to some extent, but any large-scale use will lead to the disappearance RNN problems gradient. This problem led to RNN not suitable in most real-world problems, and therefore, need to find another way to solve the context of the store.
Then the long-term short-term memory (LSTM) neural network comes in handy. Like RNN neurons, neurons LSTM stored context in its pipeline, and time to allow the order to solve the problem, and the problem does not affect the performance of the gradient disappears.
For context, the following is a typical inner working of neurons LSTM FIG. It consists of several layers and composition pointwise operations, these operations act as a data input, an output and forgotten door for LSTM **** unit status information. This is the cause of long-term memory cell state and the context of maintaining the network and the input.
2. Several forecasting methods
The first is a point by point prediction that every time we forecast only a point, this stipple out, then fill the spot just use real data to predict the next window data at a point forecast again.
The way to do sine wave is forecast to be showing such images: 
the second is the complete series prediction, we only use the first part of the training data to initialize a training window. The model then predicts the next point, and then we like to move the same point by point prediction window. The difference is that we use data from previous forecast predicted to predict. In the second step, which means that only one data point previous forecast (last point) from. In the third predicted point, the last two data points from the previous forecast, and so on. After 50 predictions, our model will be completely based on their previous forecast predicted. This allows us to use the model to predict in advance the many future point in time, but because it can be predicted predicted predicted turn-based, which will increase the error rate our forecast. As more and more of our predictions for the future, with a margin of error will be error in the previous prediction of a future prediction is increasingly being enlarged increases.
The way to do this sine wave is forecast to be rendered image: 
The third is a multi-series forecasting, it is the complete sequence of the predicted mix, because it still uses real data to initialize the test window, next to predict a point, and then use that to create a prediction point new window. But once it reaches the input window entirely by the state point in the past to predict the composition , it will stop, with reset window real data , and then start the process again. In essence, this gives the data for a plurality of trend line prediction model can be obtained to enable future trend momentum extent.
3. The not-so-simple stock market data
We can point by point estimate accurately predict hundreds of sine wave step. But we can not use this approach to be applied to the stock market, because the real world is not so simple.
With different sine wave, any specific static function of the stock market is not the time series can be mapped. Best attribute describes the stock market time series motion is random walk. As a random process, a true random walk no predictable pattern, so try to model it would be meaningless. Fortunately, many ways continues to argue that the stock market is not a purely random process , which allows us to explain theoretically the stock market time series may have some hidden mode. Because of these potential hidden mode, LSTM depth research network in the stock market have meaning.
Stock market data also involves the open, high, low, closing price and daily trading volume. It is multidimensional.
If only the closing price as input, one-dimensional model predicts made respectively using three prediction methods above, we get the following results:
Pixel Prediction: 
Sequence Prediction: 
Multiple Sequence Prediction: 
From comparison of the above results can be seen: with a point by point forecast operating data which most closely matches the real data. But a little deceptive. Because the prediction by a single line of dots to predict, based on these forecasts point of the entire previous history of the window it is real data. Therefore, in addition to the network does not know that each next point too far away from the last point, does not need too much time to understand the sequence itself. Even though it predicted the wrong point, and then the next prediction will also consider the true history and ignore the incorrect predictions, and allow errors to happen again. Although accurate predictions for the next price point of this may sound not optimistic, but it does have some important uses, such as next to accurately predict a price range or to anomaly detection.
The complete sequence prediction, It seems to be the most useless predict this type of time series (at least in this model is the use of these parameters over training). We can see that there are slight fluctuations in predicting the start, which follows the model of some kind of momentum, but soon a model converge to the equilibrium value of the time series. For predicting stock price movements, it may seem there is not much value, but the average return of traders might think that this model can be found in the average price series.
Multi-series forecasting, the network seems to correctly predict the trend (trend and magnitude) most of the time sequence. Although not perfect, but it does show the usefulness of neural networks in order LSTM depth and time series problems. By carefully adjusting the parameters of ultra certainly can achieve higher accuracy.
Next, I will use two input dimensions training model: closing price and trading volume, as follows: 
you can see two dimensions can be input so that the output predicted to become more refined. Forecast trend line seems to predict a slight decline in the coming more accurately, not only from the beginning of the main trend, the accuracy of the trend line appears to have improved in this case.
4. Summary
Currently, there are some limitations to use basic LSTM, especially in the use of financial time series, the series itself has a non-stationary characteristics is difficult to model (in terms of non-stationary time series despite solve in depth using a Bayesian neural network progress has been made). Also for some applications, also found that new progress based on neural network of the mechanism Attention has exceeded LSTM (and LSTM based Attention mechanism combined with those beyond their own). Here we offer the article project address and reference material, interested students can further explore.
Project Source Address: https://momodel.cn/explore/5d6b22c5c7e247a961bc8084?type=app
**
5. References
Blog: https://www.altumintelligence.com/articles/a/Time-Series-Prediction-Using-LSTM-Deep-Neural-Networks
videos: https://www.youtube.com/watch?v=2np77NOdnwk
Paper: LSTM, 1997 https://www.bioinf.jku.at/publications/older/2604.pdf
about us
Mo (URL: HTTPS: // momodel.cn ) is a Python support of artificial intelligence online modeling platform that can help you quickly develop, training and deployment model.
Mo AI club by artificial intelligence online modeling platform R & D and product team launched (URL: //momodel.cn: https), we are committed to the development and use artificial intelligence to reduce the threshold of the club. Team with big data processing and analysis, visualization and data modeling experience, has undertaken multidisciplinary intelligence project, with design and development capability across the board from the bottom to the front end. The main research directions for the management of large data analysis and artificial intelligence technology, and in order to promote data-driven scientific research.
Currently the team every two weeks (Saturday) held in Hangzhou line under Sharon, machine learning and related papers to share academic exchanges. Hoping to converge from all walks of life to artificial intelligence interested friends, continue to grow exchanges, promote the democratization of artificial intelligence, wider use.
