Chief Scientist of OpenAI: Facing the possibilities of AGI

The AI winter has caused differences between machine learning and AI alignment. In the 1990s, the wild imagination in the field of AI alignment was in sharp contrast with the bleak reality of machine learning. People generally held a pessimistic attitude towards the development of machine learning. Since 2010, AI technology represented by deep neural networks has developed rapidly, and AI alignment has become a research direction that has been gradually valued.

AGI is the ultimate goal of the development of AI technology. In view of the far-reaching impact of this technology, in a recent speech at the AI Alignment Forum held in San Francisco, OpenAI chief scientist Ilya Sutskever pointed out that to promote the formation of a unified understanding of AI alignment and machine learning , AGI will be It is necessary to safely incorporate machine learning into the mainstream direction. He also said that general artificial intelligence and superintelligence are likely to be realized in our lifetime and achieve huge changes. We should not limit ourselves to existing concepts and set limits on its potential.

(The following content is compiled and published by OneFlow. Please contact us for authorization for reprinting. Original text: https://www.youtube.com/watch?v=OPZxs6IXH00)

Author | Ilya Sutskever

OneFlow compilation

Translation｜Yang Ting, Wan Zilin

At present, the field of AI is developing rapidly, and general artificial intelligence (AGI) is no longer a fantasy.

Now, let me add some background knowledge to help you understand the causes of the current situation. AI alignment and machine learning have long been researched into related problems, but only recently have they begun to intersect.

AI Alignment and Machine Learning

The origins of AI alignment and AGI security can be traced back to early science fiction literature. At that time, people could still use their imagination as much as they wanted and were not limited by today's technology. People will think about the following questions: "If we have an AI system that is powerful enough, it can perform tasks such as research, science, engineering, and technology, it can conduct AI research, programming, and chip design, and it can also integrate the above capabilities together." Management, what’s going to happen?”

Asking ourselves, we might come up with the answer: That would lead to significant change. The question that naturally arises is: Such powerful AI seems to bring many challenges. As ordinary people, how should we make better use of AI? That’s why AI Alignment was born.

The field of machine learning has similar science fiction origins. In the 1940s and 1950s, some visionary scientists began to think about questions such as: What is intelligence? How does the brain work? Can we build a brain on a computer? Is it possible to create artificial neural networks? Can computers learn? Regarding the above issues, researchers at the time were very optimistic. They believed that human-level AI would be achieved within five years.

AI winter

But they were proven wrong, and then the AI winter arrived, creating a rift between machine learning and AI alignment. What are the consequences of the AI winter? It hinders the progress of machine learning and makes people pessimistic and desperate about the development prospects of machine learning.

At that time, computers were running very slowly. In this case, it was difficult for machine learning to achieve any achievements. Traditional symbolic AI became the only AI available. In the 1970s, 1980s and even 1990s, symbolic AI dominated the field of AI. With computers so slow, symbolic AI is the only option. This situation has caused great trauma to the field of AI, and people are generally pessimistic about the development of AI. At that time, it was very difficult to make progress in AI, so any progress was worthy of praise.

Pessimism about AI persisted for a long time, even into the early 2010s, when early deep learning had made progress on multiple tasks such as vision and language. I am also one of the pessimists. In my opinion, although AI has developed to a certain extent, these are temporary, development will definitely stagnate, and the AI winter will eventually return.

The above situation has led to the separation of the AI field, thus forming the two fields of AI alignment and machine learning. In the field of AI alignment, people can let their imagination run wild, ask the boldest questions about superintelligence and AI, and imagine under what circumstances AI will become better or worse. Although machine learning is very familiar with neural networks and some previous results (such as support vector machines), it can only barely classify numbers with very low resolution.

Faced with these two completely different situations, people naturally cannot connect AI alignment and machine learning. AI alignment seems too crazy and too different from the reality of machine learning. But the above view only applies to the 1990s or the beginning of the last century, and is now outdated.

As we all know, AI has developed rapidly in the 2010s and has achieved amazing results in the fields of visual recognition, translation, summary generation, game combat, and Go. In addition, technologies such as image generation and chat robots have gradually become a reality. The development of AI in recent years is likened to a high-performance car, which can accelerate from 0 to 60 yards in just two seconds.

The above cartoon, which often appears on Twitter, vividly depicts the current situation of machine learning: it is either developing too slowly or developing too fast, with only brief transitions in between, like summer on the East Coast of the United States.

Making AGI security mainstream

Now, artificial general intelligence (AGI) is no longer a bleak term. In his book Profiles of the Future, Arthur C. Clarke discusses three important technological revolutions: atomic energy, rockets, and space flight. Shortly before every breakthrough in a technological revolution, there is always an expert who asserts in a firm voice that the technology is impractical and will never become a reality.

Currently, AGI is facing a similar situation, which is quite interesting. Although AGI is still unimaginable in some aspects, it is gradually becoming a reality. Eliminate or further reduce one of the problems described in Shapes of the Future: lack of imagination . It is a lack of imagination that allows the experts mentioned above to assert that those technological revolutions will not become a reality. Their lack of imagination leads them to be overconfident in some areas. However, there is now plenty of evidence that AI is not what it once was, but in many ways we are still limited by our imaginations.

I hope everyone can solve the above problems so that we can take firm steps to incorporate AGI security into the mainstream direction of machine learning.

There are many researchers in the field of machine learning, and bringing AGI security into the mainstream of machine learning will bring even more benefits. We should combine AGI security with basic policies, so why focus on basic policies?

Why is alignment important?

The first is alignment. Many people in the machine learning field may not have been exposed to concepts related to alignment. Starting from a basic level, we will explore the issues and challenges facing alignment. Even if it does not involve specific solutions, the discussion of the above problems is valuable in itself.

I want to take a few minutes to make a few arguments to emphasize the importance of alignment. In the field of machine learning, one might ask, we’ve always been able to make AI systems do what we want them to do, so why is this changing?

Next, we'll explore different AI paradigms and explore why alignment might be harder or easier.

supervised learning

In supervised learning, we use data sets generated by human annotators for training. In speech recognition, we use human annotators to annotate speech. In machine translation, we have human translators translate the speech. In visual recognition, we use human annotators to annotate speech. , we train a neural network to imitate the behavior generated by this data.

In these cases, data understanding is relatively easy for us. We have a lot of insights into this data, so when we do supervised learning with data that we already understand well, we can worry less about the training results.

unsupervised learning

Unsupervised learning is different from the above situation for the following reasons: when we pre-train a neural network on a large amount of Internet data, we know that the neural network has acquired some knowledge about the language and the world. However, our understanding of what they learn is relatively limited and we cannot be sure what exactly they learned.

Since we know less about the behavior of neural networks, it is difficult to make neural networks achieve the behavior we expect. Empirical evidence shows that these models still have fabricated content. This is important, if it were trivial, the model wouldn't be making things up.

Therefore, unlike tasks such as speech recognition and computer vision in simple supervised learning, unsupervised learning faces new difficulties that may surprise us in an unexpected way.

This is particularly evident in Sydney, the generative AI chatbot for Microsoft's Bing search. Sydney has a rich personality that isn't entirely what Bing's creators originally intended. In the real world, there is more empirical evidence that things get more complicated once unsupervised learning begins.

The crux of the matter, of course, is that the nature of AI systems and the ease of alignment will change as the paradigm changes . The second paradigm is reinforcement learning.

reinforcement learning

Today, reinforcement learning is an integral part of the chatbots being built. After pre-training, a reinforcement learning phase is initiated, training with a reward function (or set of rewards). Perhaps we can illustrate this concretely through machine learning.

To some extent, we can indeed do this and still get good results, although we do suffer from the over-optimization issues postulated by early alignment thinkers. For example, it may be very easy to optimize a reward function or optimize a reward model learned from a human teacher, but it is also easy to learn something unexpected. The process of reinforcement learning is very complex. In fact, reinforcement learning also has a certain degree of creativity.

While we can quickly solve the problem of over-optimization, reinforcement learning faces a more critical challenge: creativity. In the field of AI, every amazing innovation comes from reinforcement learning.

For example, AlphaZero invented a completely new game strategy through reinforcement learning, a game that humans have been perfecting for thousands of years. Reinforcement learning can come up with creative solutions to problems that may be beyond our comprehension.

If AI is highly creative in interacting with the real world while also targeting outcomes we deem beneficial, what will be the consequences of reinforcement learning over medium to long-term time spans?

The problem exists, but that doesn't mean it can't be solved. In fact, it suggests that relatively simple methods can be subject to unexpected creative influences, making Sydney's antics justified.

AGI and superintelligence

Finally, let us use our imagination to explore the ultimate goal of general artificial intelligence. What happens when the AI you program produces thousands of lines of code? This will be a huge program. You can do some unit testing and even interact with the program.

This is a brand new problem that needs to be solved urgently, that is, ensuring that the 100,000 lines of code output by the AI system do not contain anything suspicious. We want to work hard to control the generation of these codes. However, this also brings a whole new challenge.

This question is not simple. It is not easy to train AI or understand its behavior when we cannot understand its output and it has strong creativity and practical action capabilities.

Now, imagine what would happen if you had an AI that could manage a company or a lab?

The last question is about deception, which by the way is a more interesting idea for those with a strict machine learning background. If there is such an artificial intelligence, it is very smart and has demonstrated superb medical capabilities during the training process, but in fact, this AI wants to be a YouTuber (video account blogger), what will happen?

Conclusion

Generally speaking, general artificial intelligence and superintelligence are possible and will probably be achieved in the short term. Although it is difficult to give an exact point in time, progress is indeed moving very quickly, so we should not limit ourselves to current ideas and place limits on its potential.

AGI will have a mega gigantic impact. In fact, "extremely gigantic" is just the conservative lower limit of AGI's influence. Faced with such a far-reaching technology, no one can predict what will happen next. Everything is possible.

Having said that, at least on a technical level, we can provide some assurance that the problem lies in human operation and not in the behavior of the technology itself. This is a relatively low base goal we can pursue.

A specific goal at the moment is that I hope to give more people a more unified understanding of AI alignment and machine learning. At present, there may be some disconnect between these two ideas, and it is of great significance to connect them and give them a more complete connotation. This is my vision for the future of AI. I hope that through discussions, exchanges and other activities, this vision will eventually become a reality.

Everyone else is watching

Welcome to Star and try OneFlow: github.com/Oneflow-Inc/oneflow/ http://github.com/Oneflow-Inc/oneflow/

Chief Scientist of OpenAI: Facing the possibilities of AGI

Guess you like