Disadvantages of purely supervised learning
What is a pre-trained model
The evolution process of the pre-training model is as follows
First academic paper written by GPT model
Everyone must have heard of the chatgpt model that became popular around the world at the end of last year. It is based on this model
Keyword Generation Painting Tool Disco Diffusion
EasyNLP: large model and small sample landing technology
Of course, this will inevitably affect the accuracy of the model, but it is a tradeoff between cost and accuracy
Parameter scale development trend
Judging from the current effect of chatgpt, the effect of increasing the parameters is still good, but the marginal benefit of increasing the parameters is also serious when the parameters are large enough to a certain extent. At this time, it may be necessary to seek a breakthrough in the algorithm or architecture
Extended model based on BERTology
First, tuning second, compression third, knowledge enhancement fourth, semantic perception fifth, language-specific sixth, multilingual and cross-lingual seventh, multimodal and cross-modal eighth, task-specific ninth, The tenth in the specific field, the eleventh in robustness, the twelfth in security, and the fusion model
Massively Distributed Parallel Training Toolkit
Aspects of comparison are as follows
Large-Scale Embedding Solution——OneEmbedding
Students who have studied computer composition principles and operating systems must be very familiar with this picture. The upper layer executes quickly, but is expensive and has a small capacity, so we introduced strategies such as cache
Correlation between sentiment and ups and downs in financial markets
January 2020 China Investor Sentiment Index
Just like the famous saying: Information is more important than gold, and investor sentiment has a great influence on market trends
investor sentiment cycle
Method of Constructing Chinese Investor Sentiment Index
The whole network collects textual big data related to investor sentiment of all listed companies. From July 2008 to May 2018, about 150 million pieces of text information have been collected. Use Chinese word segmentation technology to perform word segmentation processing on the text. Using Word2Vec technology, the words in the text are vectorized. For foreign LM dictionaries (Loughran and McDonald, 2011), translation tools are used for translation and inspection, and a Chinese version of the LM dictionary is constructed.
Among the Shanghai and Shenzhen 300 constituent stocks, select 200 stocks and select 200 discussion posts for each stock. A manual tagging team composed of professors from the National School of Development of Peking University, outstanding doctoral and master students, and market investors manually tagged these 40,000 posts. The two independently marked a piece of text information, divided it into positive, negative, and uncertain categories according to its content, and listed the positive and negative keywords contained in each post at the same time. After the labeling is completed, keep the posts with consistent labeling and classification, construct the Chinese financial sentiment dictionary (GB) according to the labeling results, and obtain the investor sentiment labeling set of the Chinese financial market. ...apply the best trained model to all text data and calculate the sentiment score of each post. Sum up the sentiment scores of different stock posts according to the corresponding standards to construct investor sentiment indexes of different index systems
FinBERT: Pretrained Financial Language Representation Model
Jane Entropy Technology FinBERT 1.0 Model
The first open-source Chinese BERT pre-training model trained on large-scale corpus in the financial field in China. Compared with the native Chinese BERT released by Google, the open-source BERT-wwm and RoBERTa-wwm-ext models of Harbin Institute of Technology Xunfei Lab, the open-source FinBERT 1.0 pre-training model has achieved remarkable results in downstream tasks in multiple financial fields. Performance improvement, without any additional adjustments, F1-score directly increased by at least 2~5.7 percentage points
Lanzhou Technology Financial Edition Mencius Model
On July 12, 2021, the Chinese language model jointly developed by the Lanzhou Technology-Innovation Works team, Shanghai Jiao Tong University, Beijing Institute of Technology and other units - the Mencius lightweight model, contains only 1 billion parameters, and is the benchmark for Chinese language understanding (Chinese Language Understanding Evaluation, CLUE) ranked first in the overall leaderboard, classification task leaderboard and reading comprehension leaderboard
Baidu Wenxin·NLP Large Model Financial Domain Model
ERNIE-Finance is trained on a large amount of financial field texts and general texts, enabling the model to learn a wealth of financial field knowledge, and improve significantly in a series of financial field tasks such as financial question answering and financial event subject extraction. The ERNIE-Finance financial domain model has learned financial domain expertise from massive financial data, and is significantly better than general models in multiple financial domain tasks. In order to improve the effect of ERNIE in the field of financial texts, ERNIE-Finance proposes a multi-data source, multi-task model branching strategy, so that the top-level structure of the model can learn financial domain knowledge during the training process, and the bottom-level structure can obtain information from financial texts and general texts at the same time. Knowledge
Investor Sentiment Index Predicts the Yield Trend of the Shanghai Composite Index
First, build an investor sentiment index based on the BERT model and stock bar comment text, and based on the search volume sentiment index of Baidu Index. Then, using LSTM-CNN in the form of two sentiment indexes and yield multi-information input to predict and analyze the positive and negative trends of the Shanghai stock index yield, within 381 trading days, 20.15% of the excess return can be obtained based on the strategy, but its The maximum retracement is as high as 5.64%
BERT Sentiment Extractor Stock Index Prediction
The author uses BERT for multi-task learning (multi-task learning, MTL), extracts the sentiment and value of news reports, and uses the measurement method of emotional polarity over time (Polarity-Over-Time, POT) to compare the news to the stock The views on the direction of the index trend are divided into five categories: very positive (very positive), positive (positive), neutral (neutral), negative (negative) and very negative (very negative), using the BERT+POT+MTL model to predict the next week stock index trend
It's not easy to create and find it helpful, please like, follow and collect~~~