This year, the biggest news in the artificial intelligence industry is the victory of DeepMind's AlphaGo against Lee Sedol. This is AlphaGo's 4-1 victory over Lee Sedol in a match against a human Go master.
The research results announced by the DeepMind team have aroused great repercussions in the AI community. It starts with deep learning (Deep Learning) and creates a new artificial intelligence technology, which is very different from the machine learning (Machine Learning) used by current artificial intelligence. Compared with traditional machine learning methods, ChatGPT allows machines to truly think like humans and is capable of many complex tasks. ChatGPT is arguably one of the most powerful language models out there.
ChatGPT is an artificial intelligence system
ChatGPT is an artificial intelligence system developed by OpenAI. It is used to simulate the deep learning model of natural language processing tasks, which can understand natural language and based on the input text content.
The ChatGPT system uses a pre-trained language model based on the Transformer architecture in the text generation process. The model is based on the Transformer architecture and uses a bidirectional Transformer model to train the language model from input text to output text, and express the input text as semantically similar Word vector, and represent the word vector as a probability distribution.
In addition, the system contains two internal neural network modules: one to handle language generation; the other to handle language understanding. In addition, the system uses a learning-based deep learning model that uses pre-trained language models to learn general language processing tasks.
How is ChatGPT trained?
DeepMind researchers built a so-called Transformer by training historical document data (Transformer is a multi-layer network composed of a deep neural network. information) model. In this model, there are three main parts: input sequence, output sequence, and weights (weights). The input sequence is some keywords in the document; the output sequence is some views on the document (depending on the topic discussed in the document); the weight is to multiply and divide the contents of the two outputs and normalize them. It can then be used to train a model that multiplies and divides the input and output and uses the results to train the Transformer model.
Since Transformer has good mathematical properties (large-scale calculations are possible), it allows GPT to complete some very complex tasks, including:
understand the text and make suggestions;
Interact with experts in multiple related fields;
What tasks can ChatGPT accomplish?
The full name of ChatGPT is Google Translate Machine Learning, and the tasks it can complete mainly include:
Conversation: Learning language and various human expressions by talking to humans. Judging from the reply content, it is mainly a question-and-answer type of dialogue;
Text Generation (Text Generation): let the machine generate text according to a specific task;
Reading Comprehension (Reading Ability): Let the machine understand the text content, for example, for a simple sentence, understand the meaning of each word in the sentence, and give reasonable answers to these words;
Translate (Translate Spontanes): Translate the text.
Why did ChatGPT "jump" to Google?
Before DeepMind released the paper, ChatGPT had already cooperated with Google, and they published their research results in "Natural Language Processing". However, the DeepMind team found that the model trained with natural language processing technology is not "smart" enough, and the performance in some specific fields is not good enough. In their paper, they proposed that they should start with more natural language processing tasks, such as translation and dialogue.
So the question is, why is ChatGPT going to Google? Of course not because Google has money, or Google has strength. In fact, these two points are not the main reason. The main reason why ChatGPT has attracted great attention and discussion within Google is the huge value it creates: helping people better understand complex scenarios such as natural language, dialogue, and question-and-answer.
Why Natural Language Processing (NLP) and Machine Learning?
The first reason is that the current artificial intelligence has entered the second half. The biggest problem in the field of AI is the limitations of data-driven artificial intelligence in solving practical problems. Now, a large amount of data is processed by computers, but many application scenarios do not actually need this data. For example, speech recognition only has an accuracy rate of more than ten percent, but it needs more than ten percent or even higher. This is because in actual scenarios, these data are not fully utilized.
The second reason is that for artificial intelligence to be truly intelligent, it needs to have human thinking ability. This thinking ability does not mean that it can have whatever ability you want it to have. If it does not have the thinking ability of humans, no matter how much data is given to it and the training model is strengthened, it will not really think like humans in the end. Therefore, natural language processing (NLP) and machine learning have become important research directions in the field of artificial intelligence.
How was ChatGPT developed, and what are the difficulties and challenges?
ChatGPT has been open-sourced on GitHub from its birth to the present, and has received support and attention from a large number of developers. So how exactly is ChatGPT developed?
ChatGPT is realized through deep learning technology. Unlike the previous training model that used a large amount of text, its biggest difficulty is to realize the transfer of knowledge on the model. As mentioned above, the model contains a lot of text information and knowledge, so ChatGPT needs to learn how to generate new text through a neural network. In practical applications, it is impossible for a machine to learn all knowledge, so it must learn from scratch, and it is difficult to generalize.
In addition, how to make the model truly capable of thinking is also one of the difficulties. To realize such a function, a large amount of data collection, data cleaning, model training and model prediction need to be solved. Most importantly, how to make the model have the ability to perceive and make reasonable judgments based on the context, these are very difficult but necessary research directions.
At the same time, ChatGPT relies heavily on a large amount of text information, how to prevent user input problems (such as repeated input of the same question) and how to provide a good experience in the application are issues that need to be resolved.
What are the development trends and application prospects of ChatGPT?
The development trend of ChatGPT is very good. It is mainly used in three aspects. First, the application of ChatGPT in intelligent question answering system. At present, some companies have begun to build an intelligent question-answering system based on ChatGPT, so that machines can answer questions like humans, making conversations more natural. Secondly, in the application of intelligent dialogue system, ChatGPT can understand the emotions and intentions of both parties in the dialogue, so as to make the dialogue more in-depth and real. Finally, it is applied in some intelligent question answering systems.
Of course, ChatGPT also has certain problems. Although it can understand dialogue, it cannot make judgments; it can answer simple questions, but it cannot understand the meaning behind the question; it can understand the meaning of text, but it cannot ask practical questions, etc.
Currently ChatGPT is mainly used in the Chinese environment. With the continuous advancement of its technology, ChatGPT will be more and more widely used in other language environments. In the future, with the further improvement of ChatGPT's performance and the expansion of its application scope, its application scope will continue to expand to other fields such as question answering systems in other language environments, such as multilingual question answering systems for natural language processing, and multilingual question answering systems for machine translation. Translation systems, multilingual chatbots for the field of chatbots, etc.
