On August 16, the WAVE SUMMIT Deep Learning Developer Conference 2023 hosted by the National Engineering Research Center for Deep Learning Technology and Application was held in Beijing. Wang Haifeng, chief technology officer of Baidu and director of the National Engineering Research Center for Deep Learning Technology and Application, gave a keynote speech. Wang Haifeng stated for the first time that the big language model has the core basic capabilities of artificial intelligence such as understanding, generation, logic, and memory, bringing the dawn of general artificial intelligence.
The number of Paddle developers has reached 8 million
The number of models exceeds 800,000
WAVE SUMMIT Deep Learning Developers Conference started in April 2019. Wang Haifeng proposed at the first conference that deep learning has strong versatility and has the characteristics of standardization, automation and modularization of industrial mass production, which will promote artificial intelligence to enter the stage of industrial mass production. In the past four years, the development of deep learning technology and applications has fully verified this point of view. The versatility of deep learning technology is becoming stronger and stronger, and the standardization, automation and modularization of deep learning platforms are becoming more and more prominent. The rise of pre-trained large models has further expanded the depth and breadth of artificial intelligence applications. Artificial intelligence has entered the stage of industrial mass production.
In terms of standardization, joint optimization of frameworks and models, unified adaptation of multiple hardware, simple and efficient application models, greatly lowering the threshold for artificial intelligence applications; in terms of automation, from training, adaptation, to inference deployment, improve the efficiency of the entire process of artificial intelligence research and development; modularization On the one hand, the rich industrial-level model library supports the convenient application of artificial intelligence in a wide range of scenarios.
It is understood that thanks to the mutual promotion of the Flying Paddle industry-level deep learning open source open platform and the Wenxin large model, the Flying Paddle ecosystem has become more and more prosperous. It has gathered 8 million developers and served 220,000 enterprises and institutions. Based on Flying Paddle, 80 million models. Wang Haifeng explained the meaningful meaning of the Chinese name "Xinghe Community" of AI Studio, a developer community of flying paddles, "Wenxin plus flying paddles, go to the galaxy lightly". Together with all the developers, with the support of Flying Paddle and Wenxin, we will build the Xinghe community and go to the stars and oceans of general artificial intelligence.
Big language models shed light on general artificial intelligence
Wang Haifeng said that artificial intelligence has a variety of typical abilities, and understanding, generation, logic, and memory are the core basic abilities. The stronger these four abilities are, the closer they are to general artificial intelligence, and the big language model has these four abilities. General artificial intelligence brings the dawn.
Specifically, the typical capabilities of artificial intelligence, such as creation, programming, problem solving, and planning, all depend on core basic capabilities such as understanding, generation, logic, and memory, with varying degrees of dependence. Taking problem solving as an example, from reading the questions, answering the questions to finally writing the answers, it requires the comprehensive use of comprehension, memory, logic and generative abilities.
How to obtain these abilities? Taking Wenxin Yiyan as an example, firstly, a pre-trained large model is obtained by fusion learning from trillions of data and hundreds of billions of knowledge. Technical advantages such as knowledge enhancement, retrieval enhancement and dialogue enhancement.
Furthermore, technological innovations such as optimization of data sources and data distribution through various strategies, basic model long-text modeling, multi-type and multi-stage supervised fine-tuning, multi-task adaptive supervised fine-tuning, multi-level and multi-granularity reward models, etc. Improve basic general ability. On the basis of retrieval enhancement and knowledge enhancement, enhance the grasp and application of world knowledge through knowledge point enhancement; through large-scale logical data construction, logical knowledge modeling, multi-granularity semantic knowledge combination and symbolic neural network, improve logical ability; Ensure the security of large models by building a comprehensive security system for data, content, models, and system security.
In terms of efficiency, through the end-to-end self-adaptive hybrid parallel training technology and the collaborative optimization of compression, inference, and service deployment, the Wenxin large model training speed has reached 3 times the original speed, and the inference speed has reached more than 30 times the original speed.
In terms of application, scene adaptation and collaborative optimization are carried out through data-driven, prompt construction, and plug-in enhancement. Wenxinyiyan has launched five major plug-ins: Baidu Search, Scroll Documents, Eyanyitu, Shuotujiehua, and Yijing Liuying, which enable the model to generate real-time and accurate information, long text summaries and Q&A, data insight and chart production, and based on Capabilities such as picture creation, question and answer, Wensheng video, etc. The plug-in mechanism expands the capability boundary of the large model and better adapts to the needs of the scene. Wang Haifeng said that in the future, Baidu will work with developers to build a plug-in ecosystem and share technological innovations.
Artificial intelligence represented by big language models is penetrating thousands of industries, accelerating industrial upgrading and economic growth. In this process, technological innovation and application implementation have formed a virtuous circle, the capabilities of understanding, generation, logic, and memory have continued to improve, the breadth and depth of industrial applications have continued to expand, and the large language model has brought the dawn of general artificial intelligence.