General stock forecast source code + model + data set based on LSTM neural network

General Stock Forecasting Model Based on Neural Network

Download address: General stock forecast source code + model + data set based on LSTM neural network

0 How to use How to use

• 1. Use getdata.py to download data, or use your own data source, put the data in the stock_daily directory
• 2. Use data_preprocess.py to preprocess the data, generate a pkl file, and place it in the pkl_handle directory (optional)
• 3. Adjust the parameters in train.py and init.py, first use predict...py to train the model, generate the model file, then use predict.py to make predictions, generate prediction results or test comparison charts

0.1 Introduction to predict.py parameters

• 1. –model: model name, currently supports lstm and transformer
• 2. –mode: mode, currently supports train, test and predict
• 3. –pkl: whether to use pkl file, currently supports 1 and 0
• 4. –pkl_queue: Whether to use the pkl queue mode to increase the training speed, currently supports 1 and 0
• 5. –test_code: test code, currently supports stock code
• 6. –test_gpu: whether to use gpu test, currently supports 1 and 0
• 7. –predict_days: number of days to predict, currently supports numbers

0.2 Introduction to some parameters in init.py Introduction to some parameters in init.py

• 1. TRAIN_WEIGHT: Training weight, currently supports numbers less than or equal to 1
• 2. SEQ_LEN: sequence length, currently supports numbers
• 3. BATCH_SIZE: Batch size, currently supports numbers
• 4. EPOCH: number of training rounds, currently supports numbers
• 5. LEARNING_RATE: Learning rate, currently supports numbers less than or equal to 1
• 6. WEIGHT_DECAY: Weight decay, currently supports numbers less than or equal to 1
• 7. SAVE_NUM_ITER: Save the model interval, currently supports numbers
• 8. SAVE_NUM_EPOCH: Save the model interval, currently supports numbers
• 9. SAVE_INTERVAL: Save the model time interval (seconds), currently supports numbers
• 10. OUTPUT_DIMENSION: Output dimension, currently supports numbers
• 11. INPUT_DIMENSION: input dimension, currently supports numbers
• 12. NUM_WORKERS: Number of threads, currently supports numbers
• 13. PKL: Whether to use pkl files, currently supports True and False
• 14. BUFFER_SIZE: buffer size, currently supports numbers
• 15. symbol: stock code, currently supports stock code or Generic.Data means all downloaded data
• 16. name_list: the name of the content that needs to be predicted
• 17. use_list: the content switch that needs to be predicted, 1 means use, 0 means not use

1 Project IntroductionProject Introduction

New

• 20230402

• 1. Modify the dataset reading method, use data queue and buffer, reduce the number of IO, and improve the training speed
• 2. Move global variables to init.py for easy modification

• 20230328

• 1. Modify the preprocessing data file format and add two fields ts_code and date for subsequent use
• 2. Modify lstm and transformer models to support mixed length input
• 3. In the transformer model, a decoder layer is added, which is expected to increase the prediction accuracy

• 20230327

• 1. Modified part of the running logic, and combined with the load pkl preprocessing file, greatly improved the training speed
• 2. Fixed a high-impact bug regarding data flow direction
• 3. try a new model
• 4. Added a new metric for evaluating how good a model is

• 20230325

• 1. Increase the data preprocessing function, and save the preprocessed queue as a pkl file to reduce IO loss
• 2. Modify unnecessary code
• 3. Simplify the logic, reduce the time responsibility, and the direction is to exchange space for time
• 4. Add common indicators to increase prediction accuracy

• 20230322

• 1. Increase output content control, you can define the content and quantity of output by yourself
• 2. Modify the read data source to a local csv file
• 3. Modify the IO logic, use multi-threading to read the csv file in the specified folder, store it in memory, train repeatedly, and reduce the number of IO
• 4. Modify lstm, transformer model
• 5. Add download data function, please use your own api token

Get the api token to download data: Get the api token to download data:

• 1. Register on the https://tushare.pro/ website and get enough points as required (until March 2023, you only need to modify the user information, it will be enough points, and it will not be determined in the future)
• 2. You can view your own api token on the https://tushare.pro/user/token page
• 3. Create an api.txt in the root directory of this project, and write the obtained api token into this file
• 4. Use this project getdata.py to automatically download daily data

The stock market is an important factor that guides the changes in the trading market. If you can grasp the trend of the stock market, it will be of great help to the investment of individuals and companies. However, the stock trend will be affected by many factors, so it is difficult to measure quantitatively from the influencing factors. But now, with the help of machine learning, it is possible to build a network, learn a certain scale of stock data, and obtain a model that can predict stock prices more accurately through network training, which can help us grasp the trend of stocks to a large extent. In this project, **LSTM (Long Short-Term Memory Network)** was built to successfully predict the trend of the stock.

First of all, in terms of data sets , we choose Shanghai Stock Exchange No. 000001, China Ping An Stock (No. SZ_000001) data set uses 2016.01.01-2019.12.31 stock data, the data content includes the date of the day, opening price, closing price, highest price, lowest price, Trading volume, turnover rate. The data set is split at a ratio of 0.1 to generate a test set. The training process uses the data from days T-99 to T as training input to predict the opening price of the stock on day T+1. (Here, I would like to thank Tushare for providing the stock daily data set, and welcome your support)

In terms of training models and results , we first adopted LSTM (Long Short-Term Memory Network), which can maintain contextual information compared to traditional neural networks, and is more conducive to stock prediction models based on the original market to predict future market. The LSTM network helped us get a good fitting result, and the loss quickly tended to 0. Afterwards, we used the Transformer Encoder part proposed to update the LSTM model for testing. However, it is found that the result is not as superior as LSTM, the error of curve fitting is larger, and the decline of loss is slower. Therefore, this project focuses on the realization of the LSTM model to predict the stock market.

2 Principles of LSTM model Principles of LSTM model

2.1 Time Series Model Time Series Model

Time series model : Time series predictive analysis is to use the characteristics of an event time in the past to predict the characteristics of the event in the future. This is a relatively complex predictive modeling problem. Unlike the prediction of the regression analysis model, the time series model depends on the order of events. The results of the input model are different after the values ​​of the same size are changed in order.

2.1 From RNN to LSTM From RNN to LSTM

RNN: The calculation result of each hidden layer of RNN is related to the current input and the result of the last hidden layer. Through this method, the calculation results of RNN have the characteristics of remembering the previous results. Among them, x is the input layer, o is the output layer, s is the hidden layer, and t refers to the number of calculations, V, W, U are the weights, and the state of the t-th hidden layer is shown in the following formula:

$$St=f(U\ast Xt+W\ast St-1)（１）$$

It can be seen that if the current hidden layer state is related to the previous n times through the RNN model, the amount of calculation needs to be increased, and the complexity increases exponentially. However, this problem can be solved by using LSTM network.

LSTM (Long Short-Term Memory Network) LSTM (Long Short-Term Memory Network):

LSTM is a special kind of RNN, which mainly solves the problem of gradient disappearance and gradient explosion in the process of long sequence training by Eileen. Compared with RNN, LSTM can perform better in long sequences.

• There are three stages inside LSTM: forget stage, select memory stage, output stage LSTM has three stages: the forget stage, the select memory stage, and the output stage.

  • **Forgetting stage: **Using calculation as a gate control to control the content that needs to be forgotten in the previous state.

  • **Selection memory stage: ** Select memory for the input, the gating signal is controlled by , and the input content is controlled by [External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly for representation.

  • **Output stage:**Determine the output content of the current state, through control, and also scale the output obtained in the previous stage.

  

3LSTM prediction model implementation for stock forecasting

1. Data set preparation

1. Data set preparation

• Dataset splitting: The dataset is split at a ratio of 0.1 to generate a test set. The training process uses the data from days T-99 to T as training input to predict the opening price of the stock on day T+1.

• Standardize the data: Both the training set and the test set need to be divided by the range by column. Inverse processing is required to obtain results after training is complete.

$$train([:,i])=(train([:,i]))-min(train[:,i])/(max(train[:,i])-min\left(train\right[:,i]))（2）$$

$$test([:,i])=(test([:,i]))-min(train[:,i])/(max(train[:,i])-min\left(train\right[:,i]))（3）$$

2. Model building

2. Model construction

Use the pytorch framework to build the LSTM model, and the parameter settings contained in torch.nn.LSTM() :

• The dimension of the input feature: input_size=dimension(dimension=8)

• The dimension of the hidden layer in LSTM: hidden_size=128

• The number of layers of the recurrent neural network: num_layers=3

• batch_first: TRUE

• Bias: bias is used by default

Fully connected layer parameter settings:

• First layer: in_features=128, out_features=16

• Second layer: in_features=16, out_features=1 (maps to a value)

3. Model training

3. Model training

• After debugging, it is determined that the learning rate lr=0.00001

• Optimization function: batch gradient descent (SGD)

• Batch size batch_size=4

• Training algebra epoch=100

• Loss function: MSELoss mean square loss function, the final training model gets MSELoss down to about 0.8.

4. Model prediction

4. Model prediction

The test set is verified using the trained model, and the average absolute percentage error (MAPELoss) is 0.04 compared with the real data, and the accuracy of the test set can be obtained at 96%.

5. Model results

5. Model results

The figure below shows the K-line display of the last 100 days of the overall data set: if the opening price of the day is lower than the closing price, it will be red, and if the opening price of the day is higher than the closing price, it will be green. The figure also shows information such as the trading volume of the day and the moving average.

The comparison chart of the test set results predicted by the LSTM model and the real results shows that the results predicted by the LSTM model are very close to the actual stock trend, so it has great reference value.

Changes in MSELoss during LSTM model training. It can be seen that as the number of training generations increases, the MSELoss of this model gradually tends to 0.

4 Conclusion

4 Conclusion

This project addresses the problem of stock market forecasting using machine learning methods. The project adopts the Shanghai Stock Exchange No. 000001 of the open source stock data center, China Ping An Stock (No. SZ_000001), and uses LSTM (long short-term memory neural network) that is more suitable for long-term sequence prediction for training. Through the training of the training set sequence, the test set By predicting the opening price, the LSTM stock forecasting model with an accuracy rate of 96% was finally obtained, which solved the problem of stock market forecasting more accurately.

During the project development process, the Transformer model proposed more recently than LSTM was also used, but the prediction effect on the test set was not good. Later analysis believed that it might be due to the encoder and the corresponding decoder layer in the general Transformer model, but in In the model of this project, the fully connected layer is used instead of the decoder, so the effect is not good. In subsequent studies, it can be further improved, and perhaps more optimized results than LSTM can be obtained.

Download address: General stock forecast source code + model + data set based on LSTM neural network