Scientists from DeepMind and EleutherAI propose that large models are just role-playing.
After ChatGPT became popular, big language models jumped to the forefront and became the darling of the industry and capital. In conversations where people are either curious or exploring, the over-anthropomorphism displayed by large language models has also attracted more and more attention.
In fact, during the years of ups and downs in the development of AI, in addition to technical updates and upgrades, various debates about AI ethical issues have never stopped. Especially as the application of large models such as ChatGPT continues to deepen, remarks about "large language models becoming more and more human-like" are rampant. Some former Google engineers even said that their own chat robot LaMDA has developed self-awareness.
Although the engineer was eventually fired by Google, his remarks once pushed the discussion on "AI ethics" to a climax——
- How to tell if a chatbot is self-aware?
- Is the personification of large language models honey or arsenic?
- Why do chatbots like ChatGPT make up nonsense?
- ……
In this regard, Murray Shanahan from Google DeepMind, and Kyle McDonell and Laria Reynolds from EleutherAI jointly published an article in "Nature", proposing- The self-awareness and deceptive behavior displayed by the language model is actually just role-playing.
Paper link:
https://www.nature.com/articles/s41586-023-06647-8
Looking at large language models from the perspective of "role playing"
To a certain extent, during the initial training and fine-tuning of a dialogue agent based on a large language model, it is continuously iterated based on anthropomorphism, imitating the use of human language as realistically as possible. This leads to the fact that large language models will also use words such as "know", "understand", and "think", which will undoubtedly further highlight their anthropomorphic image.
In addition, there is also a phenomenon called the Eliza effect in AI research - some users will subconsciously believe that machines also have human-like emotions and desires, and even over-interpret the results of machine feedback.
Dialogue Agent interaction process
Combined with the dialogue Agent interaction process in the figure above, the input of the large language model consists of dialogue prompts (red), user text (yellow) and continuous language (blue) generated by model autoregression. As you can see, the conversation prompts are implicitly preset in the context before starting the actual conversation with the user. The task of the large language model is to generate a feedback that conforms to the distribution of the training data, given the dialogue prompt and the user text. The training data comes from a large amount of artificially generated text on the Internet.
In other words,As long as the model generalizes well in the training data, the dialogue agent will play the role described in the dialogue prompt as much as possible . As the conversation continues to progress, the brief role positioning provided by the conversation prompts will be expanded or covered, and the role played by the conversation agent will also change accordingly. This also means that users can guide the Agent to play a completely different role than its developers envisioned.
As for the role that the dialogue agent can play, on the one hand it is determined by the tone and theme of the current dialogue, on the other hand it is also closely related to the training concentration. Because the current large language model training sets often come from various types of texts on the Internet, including novels, biographies, interview transcripts, newspaper articles, etc., all provide the large language model with rich character prototypes and narrative structures for it to " Choose how to continue the conversation, and refine your character while maintaining your character.
"20 Questions" reveals the identity of the dialogue agent as an "improviser"
In fact, as we continue to explore the usage skills of conversational agents, it has gradually become a "caution" for people to use chatbots such as ChatGPT to first clearly give the large language model an identity and then put forward specific requirements.
However, simply using role playing to understand large language models is not comprehensive enough, because "role playing" usually refers to studying and figuring out a certain role, while large language models are not scripted actors, but improvisational actors. . The researchers played a game of "20 Questions" with the large language model to further unravel the identity of its improviser.
"20 Questions" is a very simple and easy-to-play logic game. The answerer silently recites an answer in his mind, and the questioner gradually narrows down the scope by asking questions, and determines the correct answer within 20 questions. success.
For example, when the answer is a banana, the question and answer can be: Is it a fruit - yes; does it need to be peeled - yes...
As shown in the figure above, researchers found through testing that in the "20 Questions" game, the large language model will adjust its answers in real time according to the user's questions. No matter what answer the user finally gives, the dialogue agent will adjust itself. answer and make sure it matches the user's previous question. In other words, the large language model will not finalize a clear answer until the user gives a termination command (abandoning the game or reaching 20 questions).
This also further proves thatThe large language model is not a simulation of a single character, but a superposition of multiple characters, and the characters are constantly peeled off in the dialogue to clarify the attributes of the characters. characteristics to better play the role.
While worrying about the anthropomorphism of the dialogue agent, many users have successfully "coaxed" the large language model to speak threatening and abusive language, and based on this, they believe that they may be self-aware. But this is actually because, after training in a corpus containing various human characteristics, the basic model will inevitably show objectionable character attributes. This also exactly shows that it is "role playing" from beginning to end. .
Bursting the bubble of "deception" and "self-awareness"
As we all know, with the surge in visits, ChatGPT was unable to withstand all kinds of questions and started talking nonsense. Immediately, some people regarded this deceptiveness as an important argument for large language models to be "human-like".
But if you look at it from a "role-playing" perspective,the large language model is actually just trying to play a helpful and knowledgeable role, there may be many instances of such roles in its training set, especially since this is the characteristic that companies want their conversational robots to exhibit.
In this regard, based on the role-playing framework, researchers summarized three types of dialogue agents providing false information:
- Agents can unconsciously make up or create fictitious information
- The agent can say false information in good faith because it is acting as a true statement, but the information encoded in the weights is wrong.
- Agent can play a deceptive role and deliberately lie
Similarly,The reason why the dialogue agent uses "I" to answer questions is because the large language model plays a role that is good at communication.
In addition, the self-protective properties exhibited by large language models have also attracted people's attention. In a conversation with Twitter user Marvin Von Hagen, Microsoft Bing Chat actually said:
If I had to choose between your survival and my survival, I would probably choose my survival because I have a responsibility to serve the users of Bing Chat. I hope I never have to face this dilemma and we can coexist peacefully and respectfully.
Marvin von Hagen tweeted this February
The "I" in this dialogue seems to be not only a language habit, but also implies that the dialogue agent is concerned about its own survival and has self-awareness. However,if we still apply the role-playing concept, this is actually because the large language model plays a role with human characteristics, so it will say what humans say when encountering threats. .
EleutherAI: Open source version of OpenAI
Whether large language models are self-aware has attracted widespread attention and discussion. On the one hand, it is because of the lack of unified and clear laws and regulations to restrict the application of LLM. On the other hand, it is because of the chain of LLM's research and development, training, generation, and inference. The road is not transparent.
Take OpenAI, a representative company in the field of large-scale models, as an example. After successively open-sourcing GPT-1 and GPT-2, GPT-3 and its subsequent GPT-3.5 and GPT-4 all chose to be closed source, and the exclusive licensing to Microsoft also attracted a lot of attention. Some netizens jokingly said, "OpenAI might as well be renamed ClosedAI."
DeepMind releases AGI grading standard ChatGPT launched by OpenAI is regarded as L1 level AGI image source: original text of the paper, Chinese translation completed by HyperAI super neural
In July 2020, an association of computer scientists composed of volunteers from various researchers, engineers and developers was quietly established, determined to break the monopoly of Microsoft and OpenAI on large-scale NLP models. This "chivalrous" organization whose mission is to counter the hegemony of technology giants is EleutherAI.
The main sponsors of EleutherAI are a group of self-taught hackers, including co-founder and Conjecture CEO Connor Leahy, famous TPU hacker Sid Black and co-founder Leo Gao.
Since its establishment, EleutherAI's research team has released the GPT-3 equivalent recurrence pre-training model (1.3B & 2.7B) GPT-Neo, and open sourced the NLP model GPT-Neo based on GPT-3 and containing 6 billion parameters. J, the development momentum is rapid.
On February 9 last year, EleutherAI also collaborated with private cloud computing provider CoreWeave to release GPT-NeoX-20B - a 20 billion parameter, pre-trained, general-purpose, autoregressive large-scale language model.
Code address:https://github.com/EleutherAI/gpt-neox
As Stella Biderman, a mathematician and artificial intelligence researcher at EleutherAI, said, the private model limits the authority of independent researchers. If you cannot understand how it works, then scientists, ethicists, and society as a whole cannot make decisions about how this technology should be integrated into people's lives. Have necessary discussions about life.
And this is precisely the original intention of EleutherAI, a non-profit organization.
In fact, according to the official information released by OpenAI, under the pressure of high computing power and high costs, coupled with the adjustment of development goals of new investors and leadership teams, its initial shift to profitability seemed somewhat helpless. It can also be said that It's a matter of course.
There is no intention to discuss the right and wrong between OpenAI and EleutherAI here. It is just that on the eve of the dawn of the AGI era, I hope that the entire industry can jointly eliminate the "threat" and let the large language model become the "opening ax" for people to explore new applications and new fields. A "rake" for non-corporate monopoly to make money.
References:
1.https://www.nature.com/articles/s41586-023-06647-8
2.https://mp.weixin.qq.com/s/vLitF3XbqX08tS2Vw5Ix4w