From large language models to artificial general intelligence AGI
Back to Papers and Materials Catalog
1. Introduction
This blog is mainly based on Sections 9 and 10 of "Sparks of Artificial General Intelligence: Early experiments with GPT-4" and my own understanding of the current LLM and AGI. Give the direction of improvement when the large language model is moving towards AGI.
2.14 problems and improvement direction (dynamic continuous update)
1. Reduced model
The goal of model reduction is to use the smallest possible model to achieve the best possible performance. The current model size of the large language model is 10B-500B. AGI will inevitably become popular in the future, so the reduced model is a key research direction.
Caused problems :
- The training and use costs of the model are high. Large models with excellent performance like ChatGPT need tens of millions of dollars to train each time. When in use, the input and output of the model need to go through the entire large model once, resulting in huge usage costs.
- Models are difficult to generalize to individuals. Similar to the world's first computer, if there is no subsequent popularization of personal computers, it is believed that computers have only been used for large-scale scientific research or military purposes so far. The large size of the model has led to the research of large models mainly in the industry and a few top scientific research institutions, which not only makes it difficult to carry out research work, but also makes it impossible for the model to be extended to civilians. This also indirectly led to the difficulty of personalizing the development of the latter model.
Specific directions for improvement:
- Starting from the specific model, adopt a better neural network structure, loss function, optimizer, and word vector encoding. There are already a small number of works that have achieved certain results in this area, such as LLaMA .
- Combine with tools. In some specific scenarios, large models are still difficult to expand, such as mathematical calculations. No matter how strong a large-scale language model is, it is difficult to have the mathematical operation ability of a calculator. Therefore, instead of constantly trying to maintain performance without reducing the size of the model, it is better to study how to distribute the low-coupling capabilities of some models to other tools, and how to integrate tools with large models. For the detailed development of this direction combined with tools, see point 11.
2. Personalization
Personalization means that the model provides personalized services for each user. AGI will inevitably become popular in the future, so how to personalize the model is a key research direction.
Caused problems :
- In many scenarios, models need to provide personalized services for individuals or organizations. For example, in an educational scenario, we expect the model to understand and adapt to students' progress in learning, and give students learning assistance at different stages. At the same time, for different students, the model needs to give different teaching methods. For another example, in a daily chat scene, the output of the model for children and adults may be different. For children, the model should not output content that is not suitable for minors. At the same time, you need to pay attention to guiding the children correctly during the chat.
Specific directions for improvement:
- meta directive. This is a method given in "Sparks of Artificial General Intelligence: Early experiments with GPT-4". Its idea is to let the model know who the current user is, what he has said to him before, and what kind of output is suitable for him through the method of command input every time. Although this method has great limitations and is not effective enough, it is indeed the simplest way of thinking. This is what the current popular AutoGPT does, and it can indeed achieve some personalization capabilities.
3. Long-term memory
Long-term memory means that large models need to remember long text content to form new cognitions, or have episodic memory similar to humans . This is a very big defect of the large model, and it is also a problem that must be solved to realize AGI.
Caused problems :
- Models cannot cope with scenes that require long-term memory. For example, when using a large model to read a book, when the model reads the content of Chapter 2, the content of Chapter 1 may be forgotten. Although the maximum number of tokens supported by GPT-4 for one input has been increased to 25,000, it is still very difficult to read a book. For another example, there are already researches on the combination of robots and ChatGPT. At this time, robots have a big problem that they cannot have event memory similar to humans, and cannot remember when what happened and who they met.
Specific directions for improvement:
- Combined with database or knowledge map related technologies. At present, AutoGPT adopts this method, and it does have some long-term memory capabilities.
- Build a long-term memory model M and combine it with database technology. Similar ideas At present, Tsinghua University has done similar work ChatDB.
4. Confidence Calibration
Confidence calibration refers to the need for the model to judge whether an event is true or false. At present, LLM often outputs false facts. For example, if you ask LLM to write a poem by Li Bai, although its output is a poem, it is made up by itself instead of Li Bai. You asked him to give you a reference about the XX field, and its output was decent, but you really went to Google Scholar to find it, and found that you couldn't find the corresponding literature.
Caused problems :
- LLM cannot distinguish right from wrong. Its cognition is mainly based on the training set, and it does not think and judge what is true and what is false by itself. In the process of actual use, fidelity answers cannot be output.
Specific directions for improvement:
none
5. Continuous learning
Continuous learning refers to the need for the model to update itself based on the surrounding environment. After the model is trained once, it needs to be able to learn continuously.
Caused problems :
- The current model cannot be updated. Specifically, there are two situations. The first one, when a model is trained, assuming our computing resources have been updated, we currently need a larger model. At this time, the current situation faced by LLM is to completely retrain on a model with a larger number of parameters; second, when the model is in use, our current better way is to use the RLHF method for fine-tuning, so that the model can be certain It can be updated to a certain extent, but this method is not real-time, and the update efficiency is low and the entire model needs to be adjusted.
Specific directions for improvement:
none
6. Plans and goals
Plans and goals mean that the model needs to have the ability to formulate plans and goals.
Caused problems :
- When the problem that the model needs to deal with is complex, the model cannot formulate plans and goals to simplify the problem and effectively solve the problem. For example, when a person completes the task of traveling to Mount Hua, he needs to decompose this goal and complete it sequentially. For example, first you need to check how much money you have, then you need to search for the means of transportation to go, then you need to book a hotel, buy supplies for going up the mountain, then plan your action route, and finally go home by car. Each link in it may be divided into multiple steps. For example, when booking a hotel, you should consider the price and location, and consider how many rooms you can book. Such a complex goal requires proper planning and a series of goals to achieve.
- On the other hand, the model cannot specify the goal of improving itself, resulting in the inability to actively learn, which makes the model performance unable to be improved offline.
Specific directions for improvement:
- Instruction prompt + short-term memory: At present, AutoGPT adopts a similar method to realize it. It continuously prompts and disassembles a task through the prompt method, and finally specifies multiple sub-goals to achieve step by step. This method can achieve the planning and target capabilities of the model to a certain extent.
7. Dataset collection and generation
Dataset collection and generation means that the model needs to cooperate with appropriate dataset collection and generation methods.
Caused problems :
- Model performance is not high. Recently, large model papers such as LLaMA have shown that even for small and large models, expanding the dataset can help further improve performance. Conversely speaking, collecting or generating data in an appropriate way and further using it to train models can effectively improve the performance of large models.
Specific directions for improvement:
- For the collection of data sets, we can focus on the ability of the model to independently collect data online. Finally, let the model collect data autonomously on the network and use it for training to improve performance.
- For dataset generation, existing LLM models can be used to generate datasets to help improve performance, such as the WizardLM model or Wizardcode. The following figure shows an example of Wizardcode using LLM to generate data for the Code field. In addition, you can study how to let the model automatically generate data, and then use the generated data to train the model to continuously iteratively enhance performance. Recently facebook published a paper on image segmentation in the field of computer vision. It mentions a data generation method called data engine, which makes data generation and model training a closed loop, achieving the amazing result of "dividing everything".
8. Emotional Competence
Emotional ability means that the model needs to improve emotional ability to help improve performance.
Caused problems :
- It not only affects the usual model performance, but also makes it impossible to apply to some scenes that require emotion. For example, when chatting with someone, if the other party is a broken-hearted person. An emotional answer is bound to be better than plain reasoning or comfort. Future pet robots or robot butlers are also key research directions, and these scenarios obviously require LLM to develop emotional capabilities.
Specific directions for improvement:
none
9. Enhanced Multimodality
Enhanced multimodality means that the LLM model considers processing text, images, audio, etc. at the same time.
Caused problems :
- Existing models mainly process text, and a few can process images at the same time (GPT-4). Therefore, there is a lack of processing of audio data, for example, a large model cannot judge whether a song is good or not.
Specific directions for improvement:
- Different models are trained for different multimodalities and then integrated.
10. Integrate with existing software tools
WeChat, Taobao, Meituan, QQ Music, 12306, etc.
11. Combine with existing hardware tools
Robotic arm, bionic human skin, infrared, NFC.
12. Self-knowledge and world-knowledge
There is no confidence calibration without self-knowledge. How no world cognition eliminates prejudice.
Caused problems :
- At present, the biggest gap between LLM and AGI is still self-cognition and world cognition.
Specific directions for improvement:
none
13. AGI test questions
At present, it is more common in the way of using examinations, such as the bar qualification examination, GRE and so on.
14. Safety issues and legal issues of AGI
Skip it for now, readers can take a look at "Planing for AGI and beyond"